[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:17 dist-test-slave-n2td krb5kdc[14100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:17 dist-test-slave-n2td krb5kdc[14100](info): set up 2 sockets
Jun 19 20:44:17 dist-test-slave-n2td krb5kdc[14100](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 20:44:18.289085 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:18 dist-test-slave-n2td krb5kdc[14100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:18.409566 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36131
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:36131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:18.494208 14113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:18.494478 14113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:18.494546 14113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:18.495743 14113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:18.495818 14113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:18.495843 14113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:18.495877 14113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:18.495903 14113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:18.497421 14113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:36131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:18.497713 14113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:18.497963 14113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:18.503410 14118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:18.503510 14119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:18.503453 14121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:18.503587 14113 server_base.cc:1034] running on GCE node
I20260619 20:44:18.503816 14113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:18.504124 14113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:18.505304 14113 hybrid_clock.cc:648] HybridClock initialized: now 1781901858505290 us; error 49 us; skew 500 ppm
Jun 19 20:44:18 dist-test-slave-n2td krb5kdc[14100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901858, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:18.507803 14113 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:18.508251 14113 webserver.cc:460] Webserver started at http://127.13.195.126:43243/ using document root <none> and password file <none>
I20260619 20:44:18.508664 14113 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:18.508761 14113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:18.508925 14113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:18.510072 14113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "82b72dabd808467ca9d5623b749e086c"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "2c82742f292067591eeed9b57e6c740e"
server_key_iv: "103e25803d3136886b145ff677fc3344"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.510520 14113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "82b72dabd808467ca9d5623b749e086c"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "2c82742f292067591eeed9b57e6c740e"
server_key_iv: "103e25803d3136886b145ff677fc3344"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.511986 14113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:18.512893 14128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:18.513087 14113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:18.513195 14113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "82b72dabd808467ca9d5623b749e086c"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "2c82742f292067591eeed9b57e6c740e"
server_key_iv: "103e25803d3136886b145ff677fc3344"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.513332 14113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:18.523700 14113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:18.525878 14113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:18.526068 14113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:18.530534 14113 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:36131
I20260619 20:44:18.530562 14180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:36131 every 8 connection(s)
I20260619 20:44:18.530948 14113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:18.531855 14181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:18.534162 14181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c: Bootstrap starting.
I20260619 20:44:18.534771 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 14113
I20260619 20:44:18.534821 14181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:18.534898 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:18.535101 14181 log.cc:824] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:18.535564 14093 external_mini_cluster.cc:1428] Setting key 06a85e05030a4d7334c4f39f54465e24
I20260619 20:44:18.535857 14181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c: No bootstrap required, opened a new log
I20260619 20:44:18.537667 14181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b72dabd808467ca9d5623b749e086c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36131 } }
I20260619 20:44:18.537806 14181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:18.537829 14181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b72dabd808467ca9d5623b749e086c, State: Initialized, Role: FOLLOWER
I20260619 20:44:18.537961 14181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b72dabd808467ca9d5623b749e086c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36131 } }
I20260619 20:44:18.538033 14181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:18.538055 14181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:18.538123 14181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:18.539219 14181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b72dabd808467ca9d5623b749e086c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36131 } }
I20260619 20:44:18.539316 14181 leader_election.cc:302] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82b72dabd808467ca9d5623b749e086c; no voters:
I20260619 20:44:18.539499 14181 leader_election.cc:288] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:18.539518 14186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:18.539621 14186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [term 1 LEADER]: Becoming Leader. State: Replica: 82b72dabd808467ca9d5623b749e086c, State: Running, Role: LEADER
I20260619 20:44:18.539700 14186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b72dabd808467ca9d5623b749e086c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36131 } }
I20260619 20:44:18.539754 14181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:18.540023 14186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82b72dabd808467ca9d5623b749e086c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b72dabd808467ca9d5623b749e086c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36131 } } }
I20260619 20:44:18.540097 14186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:18.540210 14186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82b72dabd808467ca9d5623b749e086c. Latest consensus state: current_term: 1 leader_uuid: "82b72dabd808467ca9d5623b749e086c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b72dabd808467ca9d5623b749e086c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36131 } } }
I20260619 20:44:18.540293 14186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:18.540431 14191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:18.541186 14191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:18.542838 14191 catalog_manager.cc:1347] Generated new cluster ID: c729a50d03d142188d92a7c5e97481bd
I20260619 20:44:18.542924 14191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:44:18 dist-test-slave-n2td krb5kdc[14100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:18.549408 14191 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:18.549940 14191 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:18.552685 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:18.536483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.13.195.126:36131)
0619 20:44:18.536661 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:18.536671 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:18.536901 (+ 230us) server_negotiation.cc:407] Connection header received
0619 20:44:18.538842 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:18.538852 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:18.538919 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:18.539074 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:18.541401 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.542660 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.543852 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.544273 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.544411 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:18.547848 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:18.547858 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:18.547859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:18.547870 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:18.550625 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:18.551131 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:18.551134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:18.551135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:18.551250 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:18.551530 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:18.551531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:18.551531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:18.551834 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:18.552001 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:18.552276 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 20:44:18.552432 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260619 20:44:18.558418 14191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82b72dabd808467ca9d5623b749e086c: Generated new TSK 0
I20260619 20:44:18.558660 14191 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:18.585350 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36131
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:18.668045 14207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:18.668237 14207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:18.668262 14207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:18.669407 14207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:18.669486 14207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:18.669551 14207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:18.671051 14207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:18.671298 14207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:18.671536 14207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:18.676612 14216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:18.676625 14213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:18.676687 14207 server_base.cc:1034] running on GCE node
W20260619 20:44:18.676627 14212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:18.677100 14207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:18.677323 14207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:18.678504 14207 hybrid_clock.cc:648] HybridClock initialized: now 1781901858678470 us; error 64 us; skew 500 ppm
Jun 19 20:44:18 dist-test-slave-n2td krb5kdc[14100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901858, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:18.681543 14207 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:18.682003 14207 webserver.cc:460] Webserver started at http://127.13.195.65:44885/ using document root <none> and password file <none>
I20260619 20:44:18.682231 14207 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:18.682299 14207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:18.682487 14207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:18.683578 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "4db5dd7399b14cc79d74f84722e61973"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "a2d0df157c8988054582e468809ff52f"
server_key_iv: "fce6963381a9900609216d9ad634700d"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.683944 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "4db5dd7399b14cc79d74f84722e61973"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "a2d0df157c8988054582e468809ff52f"
server_key_iv: "fce6963381a9900609216d9ad634700d"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.685305 14207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:18.686108 14223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:18.686286 14207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:18.686393 14207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "4db5dd7399b14cc79d74f84722e61973"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "a2d0df157c8988054582e468809ff52f"
server_key_iv: "fce6963381a9900609216d9ad634700d"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.686524 14207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:18.711990 14207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:18.714408 14207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:18.714596 14207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:18.714852 14207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:18.715175 14207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:18.715230 14207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:18.715283 14207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:18.715328 14207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:18.722057 14207 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:46075
I20260619 20:44:18.722131 14336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:46075 every 8 connection(s)
I20260619 20:44:18.722510 14207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:44:18 dist-test-slave-n2td krb5kdc[14100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901858, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:18.730768 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 14207
I20260619 20:44:18.730862 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:18.731017 14093 external_mini_cluster.cc:1428] Setting key 88faf53f56a3a22f6fa8ce42aab5df05
I20260619 20:44:18.732508 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:18.723398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:35511 (local address 127.13.195.126:36131)
0619 20:44:18.723473 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:18.723476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:18.723625 (+ 149us) server_negotiation.cc:407] Connection header received
0619 20:44:18.724193 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:18.724194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:18.724208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:18.724270 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:18.725340 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.725871 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.726997 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.727227 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.727327 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:18.729637 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:18.729638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:18.729638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:18.729645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:18.731063 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:18.731475 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:18.731476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:18.731476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:18.731548 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:18.731802 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:18.731803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:18.731803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:18.731923 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:18.732002 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:18.732278 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 20:44:18.732415 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:44:18.732650 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:18.723266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36131 (local address 127.13.195.65:35511)
0619 20:44:18.723467 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:18.723524 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:18.724075 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:18.724360 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:18.724364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:18.725019 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:18.725218 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:18.725223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.725994 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.725996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:18.726898 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:18.726901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.727337 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.727339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:18.727468 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:18.727970 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:18.727979 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:18.729525 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:18.731147 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:18.731155 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:18.731157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:18.731386 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:18.731616 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:18.731617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:18.731618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:18.731727 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:18.732002 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:18.732011 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:18.732147 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 20:44:18.732255 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 20:44:18.732419 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260619 20:44:18.733146 14337 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36131
I20260619 20:44:18.733240 14337 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:18.733446 14337 heartbeater.cc:511] Master 127.13.195.126:36131 requested a full tablet report, sending...
I20260619 20:44:18.734040 14145 ts_manager.cc:194] Registered new tserver with Master: 4db5dd7399b14cc79d74f84722e61973 (127.13.195.65:46075)
I20260619 20:44:18.735034 14145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:35511
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:18.756897 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36131
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:18.843255 14342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:18.843477 14342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:18.843501 14342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:18.844662 14342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:18.844707 14342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:18.844734 14342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:18.846197 14342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:18.846499 14342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:18.846737 14342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:18.851718 14347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:18.851727 14349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:18.851727 14351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:18.851938 14342 server_base.cc:1034] running on GCE node
I20260619 20:44:18.852136 14342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:18.852408 14342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:18.853577 14342 hybrid_clock.cc:648] HybridClock initialized: now 1781901858853563 us; error 25 us; skew 500 ppm
Jun 19 20:44:18 dist-test-slave-n2td krb5kdc[14100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901858, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:18.856093 14342 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:18.856557 14342 webserver.cc:460] Webserver started at http://127.13.195.66:34417/ using document root <none> and password file <none>
I20260619 20:44:18.856792 14342 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:18.856861 14342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:18.857012 14342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:18.858045 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "cee7bc2c5d84470cacf48e6cd37c46df"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "a86c1f0f4d5a15fa89dbcfd61c3cd4ef"
server_key_iv: "6547fe4753f441e52d0a35a8023f4dc0"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.858474 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "cee7bc2c5d84470cacf48e6cd37c46df"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "a86c1f0f4d5a15fa89dbcfd61c3cd4ef"
server_key_iv: "6547fe4753f441e52d0a35a8023f4dc0"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.859850 14342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:18.860749 14358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:18.860961 14342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:18.861060 14342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "cee7bc2c5d84470cacf48e6cd37c46df"
format_stamp: "Formatted at 2026-06-19 20:44:18 on dist-test-slave-n2td"
server_key: "a86c1f0f4d5a15fa89dbcfd61c3cd4ef"
server_key_iv: "6547fe4753f441e52d0a35a8023f4dc0"
server_key_version: "encryptionkey@0"
I20260619 20:44:18.861150 14342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:18.889879 14342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:18.892313 14342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:18.892477 14342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:18.892715 14342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:18.893029 14342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:18.893085 14342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:18.893128 14342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:18.893168 14342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:18.900542 14342 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:45123
I20260619 20:44:18.900596 14471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:45123 every 8 connection(s)
I20260619 20:44:18.900941 14342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:18.902136 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 14342
I20260619 20:44:18.902216 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:18.902379 14093 external_mini_cluster.cc:1428] Setting key 8246352567703fd0a3f1e5fc3616fec5
Jun 19 20:44:18 dist-test-slave-n2td krb5kdc[14100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901858, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:44:18.911731 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:18.901760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:49997 (local address 127.13.195.126:36131)
0619 20:44:18.901847 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:18.901849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:18.902007 (+ 158us) server_negotiation.cc:407] Connection header received
0619 20:44:18.902689 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:18.902691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:18.902704 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:18.902770 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:18.903839 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.904600 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.905623 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.905925 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.905997 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:18.908306 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:18.908308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:18.908308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:18.908317 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:18.910059 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:18.910616 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:18.910618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:18.910618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:18.910679 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:18.910977 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:18.910978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:18.910979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:18.911122 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:18.911197 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:18.911597 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 20:44:18.911663 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:44:18.911927 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:18.901717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36131 (local address 127.13.195.66:49997)
0619 20:44:18.901878 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:18.901919 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:18.902567 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:18.902842 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:18.902846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:18.903536 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:18.903738 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:18.903744 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.904697 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.904698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:18.905515 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:18.905517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:18.906014 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:18.906015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:18.906126 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:18.906785 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:18.906797 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:18.908212 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:18.910154 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:18.910160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:18.910162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:18.910538 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:18.910780 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:18.910781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:18.910781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:18.910892 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:18.911251 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:18.911260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:18.911442 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 20:44:18.911540 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 20:44:18.911676 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260619 20:44:18.912653 14472 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36131
I20260619 20:44:18.912747 14472 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:18.912902 14472 heartbeater.cc:511] Master 127.13.195.126:36131 requested a full tablet report, sending...
I20260619 20:44:18.913383 14145 ts_manager.cc:194] Registered new tserver with Master: cee7bc2c5d84470cacf48e6cd37c46df (127.13.195.66:45123)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
I20260619 20:44:18.913829 14145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:49997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:18.929070 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36131
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:19.017961 14477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:19.018185 14477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:19.018213 14477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:19.019550 14477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:19.019600 14477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:19.019630 14477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:19.021273 14477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36131
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:19.021549 14477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:19.021797 14477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:19.027284 14484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:19.027307 14486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:19.027396 14483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:19.027432 14477 server_base.cc:1034] running on GCE node
I20260619 20:44:19.027945 14477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:19.028210 14477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:19.029361 14477 hybrid_clock.cc:648] HybridClock initialized: now 1781901859029346 us; error 32 us; skew 500 ppm
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901859, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:19.031949 14477 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:19.032361 14477 webserver.cc:460] Webserver started at http://127.13.195.67:39457/ using document root <none> and password file <none>
I20260619 20:44:19.032612 14477 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:19.032684 14477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:19.032829 14477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:19.033946 14477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "fb5a0d6366c34822926481ddccdb9892"
format_stamp: "Formatted at 2026-06-19 20:44:19 on dist-test-slave-n2td"
server_key: "2d5af254483bab56ca493acbfd6dbc1a"
server_key_iv: "6af424294d338ec0d9cdff5a90679adc"
server_key_version: "encryptionkey@0"
I20260619 20:44:19.034330 14477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "fb5a0d6366c34822926481ddccdb9892"
format_stamp: "Formatted at 2026-06-19 20:44:19 on dist-test-slave-n2td"
server_key: "2d5af254483bab56ca493acbfd6dbc1a"
server_key_iv: "6af424294d338ec0d9cdff5a90679adc"
server_key_version: "encryptionkey@0"
I20260619 20:44:19.035872 14477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:19.036830 14493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:19.037019 14477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:19.037119 14477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "fb5a0d6366c34822926481ddccdb9892"
format_stamp: "Formatted at 2026-06-19 20:44:19 on dist-test-slave-n2td"
server_key: "2d5af254483bab56ca493acbfd6dbc1a"
server_key_iv: "6af424294d338ec0d9cdff5a90679adc"
server_key_version: "encryptionkey@0"
I20260619 20:44:19.037197 14477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:19.057360 14477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:19.059953 14477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:19.060127 14477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:19.060401 14477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:19.060756 14477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:19.060797 14477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:19.060820 14477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:19.060868 14477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:19.072577 14477 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:34009
I20260619 20:44:19.072808 14606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:34009 every 8 connection(s)
I20260619 20:44:19.073128 14477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:19.074493 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 14477
I20260619 20:44:19.074596 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:19.074792 14093 external_mini_cluster.cc:1428] Setting key 0770d87e6211817ce06310e1d7479630
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901859, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:19.084573 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:19.074078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:57965 (local address 127.13.195.126:36131)
0619 20:44:19.074188 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:19.074191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:19.074586 (+ 395us) server_negotiation.cc:407] Connection header received
0619 20:44:19.075548 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:19.075549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:19.075561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:19.075614 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:19.076881 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.077463 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.078829 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.079042 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.079142 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:19.081639 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:19.081640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:19.081641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:19.081649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:19.083130 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:19.083574 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:19.083576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:19.083576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:19.083637 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:19.083907 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:19.083908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:19.083909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:19.084028 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:19.084107 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:19.084389 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 20:44:19.084477 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:44:19.084815 14609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:19.074050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36131 (local address 127.13.195.67:57965)
0619 20:44:19.074346 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:19.074403 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:19.075437 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:19.075692 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:19.075696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:19.076466 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:19.076756 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:19.076761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.077567 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.077568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:19.078697 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:19.078700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.079165 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.079167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:19.079350 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:19.079960 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:19.079969 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:19.081524 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:19.083233 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:19.083240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:19.083243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:19.083484 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:19.083705 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:19.083706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:19.083707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:19.083824 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:19.084099 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:19.084105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:19.084248 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:44:19.084366 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 20:44:19.084525 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":128,"threads_started":1}
I20260619 20:44:19.085494 14607 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36131
I20260619 20:44:19.085590 14607 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:19.085734 14607 heartbeater.cc:511] Master 127.13.195.126:36131 requested a full tablet report, sending...
I20260619 20:44:19.086117 14145 ts_manager.cc:194] Registered new tserver with Master: fb5a0d6366c34822926481ddccdb9892 (127.13.195.67:34009)
I20260619 20:44:19.086552 14145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:57965
I20260619 20:44:19.087327 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901859, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901859, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:19.111537 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:19.102864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.13.195.65:46075)
0619 20:44:19.103078 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:19.103084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:19.103097 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:44:19.103123 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:19.103125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:19.103139 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:19.103210 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:19.103931 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.104882 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.105679 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.106052 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.106109 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:19.108614 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:19.108623 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:19.108623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:19.108630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:19.110308 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:19.110720 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:19.110722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:19.110723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:19.110790 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:19.111025 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:19.111025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:19.111026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:19.111155 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:19.111304 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:19.111375 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 20:44:19.111442 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":113,"threads_started":1}
W20260619 20:44:19.112028 14251 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38080
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:19.136039 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:19.128604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.13.195.65:46075)
0619 20:44:19.128688 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:19.128691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:19.128789 (+ 98us) server_negotiation.cc:407] Connection header received
0619 20:44:19.128890 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:19.128891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:19.128905 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:19.128978 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:19.129589 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.130088 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.130967 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:19.131172 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:19.131251 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:19.133616 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:19.133617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:19.133618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:19.133625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:19.134895 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:19.135238 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:19.135239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:19.135239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:19.135295 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:19.135495 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:19.135496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:19.135497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:19.135595 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:19.135800 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:19.135846 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 20:44:19.135925 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 20:44:19.136605 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 14207
I20260619 20:44:19.142256 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 14342
I20260619 20:44:19.147703 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 14477
I20260619 20:44:19.151871 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 14113
2026-06-19T20:44:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1912 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14621](info): set up 2 sockets
Jun 19 20:44:19 dist-test-slave-n2td krb5kdc[14621](info): commencing operation
krb5kdc: starting...
W20260619 20:44:20.201222 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:20.316979 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:39647
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:39647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:20.397040 14634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:20.397305 14634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:20.397341 14634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:20.398509 14634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:20.398579 14634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:20.398602 14634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:20.398624 14634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:20.398653 14634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:20.400111 14634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:39647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:39647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:20.400399 14634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:20.400628 14634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:20.405707 14639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.405725 14640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.405728 14642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:20.405956 14634 server_base.cc:1034] running on GCE node
I20260619 20:44:20.406324 14634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:20.406627 14634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:20.407769 14634 hybrid_clock.cc:648] HybridClock initialized: now 1781901860407742 us; error 47 us; skew 500 ppm
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:20.410475 14634 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:20.410912 14634 webserver.cc:460] Webserver started at http://127.13.195.126:45419/ using document root <none> and password file <none>
I20260619 20:44:20.411134 14634 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:20.411183 14634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:20.411341 14634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:20.412501 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "007e322ab7f240028ef66455bc3092e6"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "5057d3eb41567c280c6b119840871d0b"
server_key_iv: "9bad0167ef67fd4109fe26ec19074b02"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.412916 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "007e322ab7f240028ef66455bc3092e6"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "5057d3eb41567c280c6b119840871d0b"
server_key_iv: "9bad0167ef67fd4109fe26ec19074b02"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.414418 14634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:20.415340 14649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.415539 14634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:20.415621 14634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "007e322ab7f240028ef66455bc3092e6"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "5057d3eb41567c280c6b119840871d0b"
server_key_iv: "9bad0167ef67fd4109fe26ec19074b02"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.415673 14634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:20.456918 14634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:20.459237 14634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:20.459415 14634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:20.463923 14634 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:39647
I20260619 20:44:20.463958 14702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:39647 every 8 connection(s)
I20260619 20:44:20.464399 14634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:20.465291 14703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:20.467603 14703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6: Bootstrap starting.
I20260619 20:44:20.468197 14703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:20.468483 14703 log.cc:824] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:20.469213 14703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6: No bootstrap required, opened a new log
I20260619 20:44:20.470903 14703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007e322ab7f240028ef66455bc3092e6" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39647 } }
I20260619 20:44:20.471086 14703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:20.471138 14703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 007e322ab7f240028ef66455bc3092e6, State: Initialized, Role: FOLLOWER
I20260619 20:44:20.471293 14703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007e322ab7f240028ef66455bc3092e6" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39647 } }
I20260619 20:44:20.471378 14703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:20.471421 14703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:20.471469 14703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:20.471987 14703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007e322ab7f240028ef66455bc3092e6" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39647 } }
I20260619 20:44:20.472102 14703 leader_election.cc:302] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 007e322ab7f240028ef66455bc3092e6; no voters:
I20260619 20:44:20.472168 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 14634
I20260619 20:44:20.472304 14703 leader_election.cc:288] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:20.472316 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:20.472433 14706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:20.472513 14093 external_mini_cluster.cc:1428] Setting key 7a7df9c16b7c560226413bb26aad3721
I20260619 20:44:20.472553 14703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:20.472607 14706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [term 1 LEADER]: Becoming Leader. State: Replica: 007e322ab7f240028ef66455bc3092e6, State: Running, Role: LEADER
I20260619 20:44:20.472739 14706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007e322ab7f240028ef66455bc3092e6" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39647 } }
I20260619 20:44:20.473256 14707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "007e322ab7f240028ef66455bc3092e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007e322ab7f240028ef66455bc3092e6" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39647 } } }
I20260619 20:44:20.473358 14707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:20.473596 14709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 007e322ab7f240028ef66455bc3092e6. Latest consensus state: current_term: 1 leader_uuid: "007e322ab7f240028ef66455bc3092e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "007e322ab7f240028ef66455bc3092e6" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39647 } } }
I20260619 20:44:20.473735 14709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:20.474009 14718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:20.474650 14718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:20.476305 14718 catalog_manager.cc:1347] Generated new cluster ID: 0a8a93a1d2664e468c7ca50651a3fdd2
I20260619 20:44:20.476383 14718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:20.483966 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:20.473215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.13.195.126:39647)
0619 20:44:20.473408 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:20.473418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:20.473452 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:44:20.473580 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:20.473588 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:20.473649 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:20.473800 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:20.474840 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.476123 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.477074 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.477543 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.477686 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:20.480274 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:20.480278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:20.480279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:20.480299 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:20.482141 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:20.482643 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:20.482646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:20.482647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:20.482770 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:20.483010 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:20.483011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:20.483012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:20.483281 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:20.483481 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:20.483595 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 20:44:20.483750 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260619 20:44:20.511233 14718 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:20.511929 14718 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:20.519058 14718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 007e322ab7f240028ef66455bc3092e6: Generated new TSK 0
I20260619 20:44:20.519299 14718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:20.556746 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:39647
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:20.665269 14730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:20.665486 14730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:20.665510 14730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:20.666642 14730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:20.666683 14730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:20.666709 14730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:20.668140 14730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:39647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:20.668325 14730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:20.668519 14730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:20.673625 14735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.673604 14736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.673609 14738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:20.673910 14730 server_base.cc:1034] running on GCE node
I20260619 20:44:20.674157 14730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:20.674405 14730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:20.675599 14730 hybrid_clock.cc:648] HybridClock initialized: now 1781901860675583 us; error 34 us; skew 500 ppm
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:20.678090 14730 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:20.678606 14730 webserver.cc:460] Webserver started at http://127.13.195.65:35133/ using document root <none> and password file <none>
I20260619 20:44:20.678835 14730 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:20.678900 14730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:20.679035 14730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:20.680059 14730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "297854506d68483bb8b94da0b5518749"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "bfc6d1ed9275a2dc4dbac252ec30c5db"
server_key_iv: "0c448d2326952c0dad5a86a80cbdf32b"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.680424 14730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "297854506d68483bb8b94da0b5518749"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "bfc6d1ed9275a2dc4dbac252ec30c5db"
server_key_iv: "0c448d2326952c0dad5a86a80cbdf32b"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.681725 14730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:20.682500 14745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.682693 14730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:20.682790 14730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "297854506d68483bb8b94da0b5518749"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "bfc6d1ed9275a2dc4dbac252ec30c5db"
server_key_iv: "0c448d2326952c0dad5a86a80cbdf32b"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.682876 14730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:20.695341 14730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:20.697353 14730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:20.697528 14730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:20.697773 14730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:20.698091 14730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:20.698148 14730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.698184 14730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:20.698210 14730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.704298 14730 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:39439
I20260619 20:44:20.704443 14861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:39439 every 8 connection(s)
I20260619 20:44:20.704716 14730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:20.712689 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 14730
I20260619 20:44:20.712805 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:20.713025 14093 external_mini_cluster.cc:1428] Setting key 95ecfbc7b85f88f66790e878c61aeff1
I20260619 20:44:20.714350 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:20.705510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:57071 (local address 127.13.195.126:39647)
0619 20:44:20.705584 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:20.705587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:20.705789 (+ 202us) server_negotiation.cc:407] Connection header received
0619 20:44:20.706741 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:20.706742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:20.706752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:20.706823 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:20.707396 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.707897 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.709011 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.709206 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.709263 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:20.711334 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:20.711336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:20.711336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:20.711343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:20.712777 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:20.713302 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:20.713303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:20.713304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:20.713412 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:20.713683 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:20.713684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:20.713684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:20.713807 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:20.713885 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:20.714162 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 20:44:20.714251 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:44:20.714566 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:20.705492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:39647 (local address 127.13.195.65:57071)
0619 20:44:20.705655 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:20.705699 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:20.706640 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:20.706876 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:20.706879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:20.707132 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:20.707324 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:20.707328 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.708034 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.708035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:20.708932 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:20.708935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.709281 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.709282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:20.709378 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:20.709846 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:20.709853 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:20.711229 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:20.712899 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:20.712907 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:20.712909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:20.713213 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:20.713486 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:20.713487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:20.713488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:20.713611 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:20.713876 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:20.713883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:20.714028 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 20:44:20.714141 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 20:44:20.714299 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260619 20:44:20.715464 14862 heartbeater.cc:347] Connected to a master server at 127.13.195.126:39647
I20260619 20:44:20.715559 14862 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:20.715754 14862 heartbeater.cc:511] Master 127.13.195.126:39647 requested a full tablet report, sending...
I20260619 20:44:20.716270 14666 ts_manager.cc:194] Registered new tserver with Master: 297854506d68483bb8b94da0b5518749 (127.13.195.65:39439)
I20260619 20:44:20.716904 14666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:57071
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:20.737332 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:39647
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:20.819252 14867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:20.819509 14867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:20.819550 14867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:20.820748 14867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:20.820822 14867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:20.820870 14867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:20.822420 14867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:39647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:20.822739 14867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:20.822985 14867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:20.828186 14876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.828263 14874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.828186 14873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:20.828531 14867 server_base.cc:1034] running on GCE node
I20260619 20:44:20.828768 14867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:20.829041 14867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:20.830193 14867 hybrid_clock.cc:648] HybridClock initialized: now 1781901860830182 us; error 27 us; skew 500 ppm
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:20.832872 14867 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:20.833333 14867 webserver.cc:460] Webserver started at http://127.13.195.66:39735/ using document root <none> and password file <none>
I20260619 20:44:20.833520 14867 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:20.833560 14867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:20.833653 14867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:20.834769 14867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "57490e6474a34cee8c281b1654785252"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "cd2944f2be96a7056b66a63b4d4f685b"
server_key_iv: "f46090ba88ff27507edffa88f36e2b81"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.835078 14867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "57490e6474a34cee8c281b1654785252"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "cd2944f2be96a7056b66a63b4d4f685b"
server_key_iv: "f46090ba88ff27507edffa88f36e2b81"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.836491 14867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:20.837250 14883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.837471 14867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:20.837543 14867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "57490e6474a34cee8c281b1654785252"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "cd2944f2be96a7056b66a63b4d4f685b"
server_key_iv: "f46090ba88ff27507edffa88f36e2b81"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.837651 14867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:20.850787 14867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:20.853129 14867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:20.853283 14867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:20.853538 14867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:20.853853 14867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:20.853907 14867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.853942 14867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:20.853971 14867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.860041 14867 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:33843
I20260619 20:44:20.860085 14996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:33843 every 8 connection(s)
I20260619 20:44:20.860406 14867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:20.862046 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 14867
I20260619 20:44:20.862131 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:20.862367 14093 external_mini_cluster.cc:1428] Setting key e7036ed894bc8d2f414c8c1167654271
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:44:20.870507 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:20.861114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:50827 (local address 127.13.195.126:39647)
0619 20:44:20.861161 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:20.861164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:20.861355 (+ 191us) server_negotiation.cc:407] Connection header received
0619 20:44:20.862322 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:20.862324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:20.862341 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:20.862479 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:20.863101 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.863598 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.864890 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.865080 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.865146 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:20.867598 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:20.867599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:20.867600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:20.867607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:20.869032 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:20.869480 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:20.869481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:20.869482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:20.869546 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:20.869817 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:20.869818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:20.869818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:20.869945 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:20.870022 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:20.870294 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 20:44:20.870383 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:20.870826 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:20.861070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:39647 (local address 127.13.195.66:50827)
0619 20:44:20.861225 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:20.861266 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:20.862206 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:20.862542 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:20.862548 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:20.862811 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:20.863001 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:20.863005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.863703 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.863704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:20.864779 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:20.864782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:20.865190 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:20.865191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:20.865336 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:20.865898 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:20.865908 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:20.867487 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:20.869138 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:20.869145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:20.869148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:20.869399 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:20.869615 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:20.869616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:20.869616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:20.869731 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:20.870013 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:20.870019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:20.870160 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 20:44:20.870273 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 20:44:20.870461 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260619 20:44:20.871381 14997 heartbeater.cc:347] Connected to a master server at 127.13.195.126:39647
I20260619 20:44:20.871486 14997 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:20.871665 14997 heartbeater.cc:511] Master 127.13.195.126:39647 requested a full tablet report, sending...
I20260619 20:44:20.872030 14666 ts_manager.cc:194] Registered new tserver with Master: 57490e6474a34cee8c281b1654785252 (127.13.195.66:33843)
I20260619 20:44:20.872506 14666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:50827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:20.888738 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:39647
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:20.972836 15002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:20.973044 15002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:20.973070 15002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:20.974627 15002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:20.974678 15002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:20.974709 15002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:20.976236 15002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:39647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:20.976490 15002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:20.976732 15002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:20.982012 15008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.982024 15010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:20.982031 15007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:20.982509 15002 server_base.cc:1034] running on GCE node
I20260619 20:44:20.982733 15002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:20.982945 15002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:20.984094 15002 hybrid_clock.cc:648] HybridClock initialized: now 1781901860984095 us; error 43 us; skew 500 ppm
Jun 19 20:44:20 dist-test-slave-n2td krb5kdc[14621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:20.986858 15002 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:20.987388 15002 webserver.cc:460] Webserver started at http://127.13.195.67:45271/ using document root <none> and password file <none>
I20260619 20:44:20.987632 15002 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:20.987676 15002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:20.987797 15002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:20.988932 15002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "163f365a166f40f0b7cb467c14594db0"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "d6622e95af87eecf170c5adccacd8e3e"
server_key_iv: "24d525dab372bfc507e33b73c6abf4e8"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.989302 15002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "163f365a166f40f0b7cb467c14594db0"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "d6622e95af87eecf170c5adccacd8e3e"
server_key_iv: "24d525dab372bfc507e33b73c6abf4e8"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.990751 15002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:44:20.991565 15017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:20.991742 15002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:20.991833 15002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "163f365a166f40f0b7cb467c14594db0"
format_stamp: "Formatted at 2026-06-19 20:44:20 on dist-test-slave-n2td"
server_key: "d6622e95af87eecf170c5adccacd8e3e"
server_key_iv: "24d525dab372bfc507e33b73c6abf4e8"
server_key_version: "encryptionkey@0"
I20260619 20:44:20.991917 15002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:21.006078 15002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:21.008515 15002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:21.008684 15002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:21.008924 15002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:21.009232 15002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:21.009295 15002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:21.009338 15002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:21.009366 15002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:21.015671 15002 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:39047
I20260619 20:44:21.015710 15132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:39047 every 8 connection(s)
I20260619 20:44:21.016036 15002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[14621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:21.023810 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 15002
I20260619 20:44:21.023914 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:21.024117 14093 external_mini_cluster.cc:1428] Setting key fc4804bf85adc4e53d2670f6e0e7a414
I20260619 20:44:21.025956 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.016714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:36869 (local address 127.13.195.126:39647)
0619 20:44:21.016779 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.016782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.017218 (+ 436us) server_negotiation.cc:407] Connection header received
0619 20:44:21.018263 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.018264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.018274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.018325 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:21.019097 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.019740 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.020739 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.020922 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.020980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.023100 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:21.023101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:21.023102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:21.023108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:21.024737 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.025195 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.025196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.025196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.025246 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.025430 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.025431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.025431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.025540 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:21.025590 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.025832 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.025894 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:44:21.026190 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.016701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:39647 (local address 127.13.195.67:36869)
0619 20:44:21.017088 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:21.017127 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:21.018165 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:21.018389 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:21.018411 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:21.018758 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:21.018993 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.018999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.019817 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.019818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.020651 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.020653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.020998 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.020998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.021094 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.021565 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:21.021574 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:21.023018 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:21.024815 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:21.024822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:21.024824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:21.025124 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:21.025288 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:21.025288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:21.025289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:21.025380 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:21.025606 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:21.025610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:21.025723 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 20:44:21.025807 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:44:21.025920 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":151,"threads_started":1}
I20260619 20:44:21.026650 15133 heartbeater.cc:347] Connected to a master server at 127.13.195.126:39647
I20260619 20:44:21.026743 15133 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:21.026878 15133 heartbeater.cc:511] Master 127.13.195.126:39647 requested a full tablet report, sending...
I20260619 20:44:21.027226 14666 ts_manager.cc:194] Registered new tserver with Master: 163f365a166f40f0b7cb467c14594db0 (127.13.195.67:39047)
I20260619 20:44:21.027664 14666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:36869
I20260619 20:44:21.036042 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:44:21.044238 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.037632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.13.195.126:39647)
0619 20:44:21.037726 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.037728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.037763 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:44:21.037842 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.037843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.037852 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.037905 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:21.038516 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.039029 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.039794 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.039996 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.040067 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.041390 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:21.041391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:21.041392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:21.041399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:21.042955 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.043299 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.043300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.043300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.043393 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.043602 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.043608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.043608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.043771 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:21.043861 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.043960 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.044027 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 20:44:21.046654 14666 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:21.048236 14666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:21.055516 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.050924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.13.195.66:33843)
0619 20:44:21.051261 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.051266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.051278 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:44:21.051379 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.051382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.051397 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.051461 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:21.051653 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.052742 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.054526 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.055035 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.055141 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.055212 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.055391 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.055435 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":95,"threads_started":1}
I20260619 20:44:21.055508 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.050816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:33843 (local address 127.0.0.1:42814)
0619 20:44:21.051213 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:21.051224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:21.051268 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:21.051544 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:21.051544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:21.051546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:21.051579 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.051580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.052868 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.052868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.054187 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.054191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.055146 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.055147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.055229 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.055244 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:44:21.055325 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:44:21.055386 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":212,"thread_start_us":90,"threads_started":1}
I20260619 20:44:21.055517 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.050736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:39047 (local address 127.0.0.1:55376)
0619 20:44:21.050940 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:21.050985 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:21.051062 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:21.051385 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:21.051388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:21.051391 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:21.051466 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.051468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.052792 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.052793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.054053 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.054055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.055232 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.055233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.055308 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.055315 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:44:21.055380 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:44:21.055418 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":61,"thread_start_us":58,"threads_started":1}
I20260619 20:44:21.055739 15146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.050933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.13.195.67:39047)
0619 20:44:21.051121 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.051126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.051137 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:44:21.051177 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.051179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.051195 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.051284 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:21.051544 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.052673 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.054151 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.055040 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.055128 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.055191 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.055555 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.055601 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260619 20:44:21.056253 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.051245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.13.195.65:39439)
0619 20:44:21.051476 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.051481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.051491 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:21.051622 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.051624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.051642 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.051738 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:21.051968 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.052723 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.054280 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.055071 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.055135 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.055191 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.055468 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.055512 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
I20260619 20:44:21.056422 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.051010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39439 (local address 127.0.0.1:49534)
0619 20:44:21.051335 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:21.051346 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:21.051399 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:21.051845 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:21.051846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:21.051847 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:21.051884 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.051885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.052865 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.052866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.054201 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.054203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.055234 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.055235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.055308 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.055313 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:44:21.055372 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:44:21.055418 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":70,"thread_start_us":58,"threads_started":1}
I20260619 20:44:21.056314 14931 tablet_service.cc:1467] Processing CreateTablet for tablet fb1822d62d26419690619f7df3717878 (DEFAULT_TABLE table=test-table [id=43d34a1fdbdf4e88ad12c8f7a59c2c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:21.056737 14931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb1822d62d26419690619f7df3717878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:21.056768 15066 tablet_service.cc:1467] Processing CreateTablet for tablet fb1822d62d26419690619f7df3717878 (DEFAULT_TABLE table=test-table [id=43d34a1fdbdf4e88ad12c8f7a59c2c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:21.057068 15066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb1822d62d26419690619f7df3717878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:21.057184 14795 tablet_service.cc:1467] Processing CreateTablet for tablet fb1822d62d26419690619f7df3717878 (DEFAULT_TABLE table=test-table [id=43d34a1fdbdf4e88ad12c8f7a59c2c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:21.057484 14795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb1822d62d26419690619f7df3717878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:21.059886 15150 tablet_bootstrap.cc:492] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0: Bootstrap starting.
I20260619 20:44:21.060249 15151 tablet_bootstrap.cc:492] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252: Bootstrap starting.
I20260619 20:44:21.060693 15152 tablet_bootstrap.cc:492] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749: Bootstrap starting.
I20260619 20:44:21.060917 15150 tablet_bootstrap.cc:654] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:21.060946 15151 tablet_bootstrap.cc:654] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:21.061290 15152 tablet_bootstrap.cc:654] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:21.061591 15150 log.cc:824] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:21.061591 15151 log.cc:824] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:21.061800 15152 log.cc:824] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:21.062497 15151 tablet_bootstrap.cc:492] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252: No bootstrap required, opened a new log
I20260619 20:44:21.062510 15150 tablet_bootstrap.cc:492] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0: No bootstrap required, opened a new log
I20260619 20:44:21.062578 15150 ts_tablet_manager.cc:1397] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:44:21.062578 15151 ts_tablet_manager.cc:1397] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:21.062938 15152 tablet_bootstrap.cc:492] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749: No bootstrap required, opened a new log
I20260619 20:44:21.063023 15152 ts_tablet_manager.cc:1397] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:21.064586 15152 raft_consensus.cc:348] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.064770 15152 raft_consensus.cc:374] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:21.064822 15152 raft_consensus.cc:729] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297854506d68483bb8b94da0b5518749, State: Initialized, Role: FOLLOWER
I20260619 20:44:21.064941 15152 consensus_queue.cc:260] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.065042 15150 raft_consensus.cc:348] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.065078 15151 raft_consensus.cc:348] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.065239 14862 heartbeater.cc:503] Master 127.13.195.126:39647 was elected leader, sending a full tablet report...
I20260619 20:44:21.065255 15150 raft_consensus.cc:374] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:21.065255 15151 raft_consensus.cc:374] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:21.065276 15152 ts_tablet_manager.cc:1428] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:21.065297 15150 raft_consensus.cc:729] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163f365a166f40f0b7cb467c14594db0, State: Initialized, Role: FOLLOWER
I20260619 20:44:21.065297 15151 raft_consensus.cc:729] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57490e6474a34cee8c281b1654785252, State: Initialized, Role: FOLLOWER
I20260619 20:44:21.065420 15151 consensus_queue.cc:260] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.065493 15150 consensus_queue.cc:260] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.065781 14997 heartbeater.cc:503] Master 127.13.195.126:39647 was elected leader, sending a full tablet report...
I20260619 20:44:21.065798 15150 ts_tablet_manager.cc:1428] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:44:21.065917 15133 heartbeater.cc:503] Master 127.13.195.126:39647 was elected leader, sending a full tablet report...
I20260619 20:44:21.066037 15151 ts_tablet_manager.cc:1428] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260619 20:44:21.111001 14998 tablet.cc:2369] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:44:21.205506 14863 tablet.cc:2369] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:44:21.266631 15134 tablet.cc:2369] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:21.337424 15158 raft_consensus.cc:482] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:21.337553 15158 raft_consensus.cc:504] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.338017 15158 leader_election.cc:288] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57490e6474a34cee8c281b1654785252 (127.13.195.66:33843), 297854506d68483bb8b94da0b5518749 (127.13.195.65:39439)
I20260619 20:44:21.341125 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.338245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:33843 (local address 127.13.195.67:50873)
0619 20:44:21.338307 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:21.338321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:21.338412 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:21.338630 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:21.338631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:21.338633 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:21.338733 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.338734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.339436 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.339437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.340258 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.340260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.340838 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.340839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.340936 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.340939 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:21.341017 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:44:21.341055 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 20:44:21.341125 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.338309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:50873 (local address 127.13.195.66:33843)
0619 20:44:21.338417 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.338419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.338473 (+ 54us) server_negotiation.cc:407] Connection header received
0619 20:44:21.338505 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.338506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.338516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.338621 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:21.338862 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.339342 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.340356 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.340747 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.340854 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.340864 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.341024 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.341058 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 20:44:21.341578 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.338502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:50279 (local address 127.13.195.65:39439)
0619 20:44:21.338560 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.338562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.338656 (+ 94us) server_negotiation.cc:407] Connection header received
0619 20:44:21.338727 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.338728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.338738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.338845 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:21.339035 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.339503 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.340735 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.341163 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.341249 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.341259 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.341466 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.341511 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:44:21.341599 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.338374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39439 (local address 127.13.195.67:50279)
0619 20:44:21.338585 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:21.338596 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:21.338667 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:21.338855 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:21.338856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:21.338857 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:21.338962 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.338964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.339586 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.339587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.340642 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:21.340644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.341314 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.341315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:21.341388 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.341391 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:21.341444 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:44:21.341476 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":120,"threads_started":1}
I20260619 20:44:21.341569 14951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1822d62d26419690619f7df3717878" candidate_uuid: "163f365a166f40f0b7cb467c14594db0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57490e6474a34cee8c281b1654785252" is_pre_election: true
I20260619 20:44:21.341743 14951 raft_consensus.cc:2454] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163f365a166f40f0b7cb467c14594db0 in term 0.
I20260619 20:44:21.341996 15020 leader_election.cc:302] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163f365a166f40f0b7cb467c14594db0, 57490e6474a34cee8c281b1654785252; no voters:
I20260619 20:44:21.342123 15158 raft_consensus.cc:2781] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:21.342200 15158 raft_consensus.cc:482] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:21.342235 15158 raft_consensus.cc:3037] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:21.342386 14815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1822d62d26419690619f7df3717878" candidate_uuid: "163f365a166f40f0b7cb467c14594db0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297854506d68483bb8b94da0b5518749" is_pre_election: true
I20260619 20:44:21.342564 14815 raft_consensus.cc:2454] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163f365a166f40f0b7cb467c14594db0 in term 0.
I20260619 20:44:21.343202 15158 raft_consensus.cc:504] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.343303 15158 leader_election.cc:288] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [CANDIDATE]: Term 1 election: Requested vote from peers 57490e6474a34cee8c281b1654785252 (127.13.195.66:33843), 297854506d68483bb8b94da0b5518749 (127.13.195.65:39439)
I20260619 20:44:21.343526 14951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1822d62d26419690619f7df3717878" candidate_uuid: "163f365a166f40f0b7cb467c14594db0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57490e6474a34cee8c281b1654785252"
I20260619 20:44:21.343596 14951 raft_consensus.cc:3037] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:21.343568 14815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb1822d62d26419690619f7df3717878" candidate_uuid: "163f365a166f40f0b7cb467c14594db0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297854506d68483bb8b94da0b5518749"
I20260619 20:44:21.343667 14815 raft_consensus.cc:3037] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:21.344399 14951 raft_consensus.cc:2454] T fb1822d62d26419690619f7df3717878 P 57490e6474a34cee8c281b1654785252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163f365a166f40f0b7cb467c14594db0 in term 1.
I20260619 20:44:21.344524 14815 raft_consensus.cc:2454] T fb1822d62d26419690619f7df3717878 P 297854506d68483bb8b94da0b5518749 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163f365a166f40f0b7cb467c14594db0 in term 1.
I20260619 20:44:21.344559 15020 leader_election.cc:302] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163f365a166f40f0b7cb467c14594db0, 57490e6474a34cee8c281b1654785252; no voters:
I20260619 20:44:21.344678 15158 raft_consensus.cc:2781] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:21.344848 15158 raft_consensus.cc:686] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [term 1 LEADER]: Becoming Leader. State: Replica: 163f365a166f40f0b7cb467c14594db0, State: Running, Role: LEADER
I20260619 20:44:21.344951 15158 consensus_queue.cc:237] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } }
I20260619 20:44:21.345760 14664 catalog_manager.cc:5495] T fb1822d62d26419690619f7df3717878 P 163f365a166f40f0b7cb467c14594db0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 163f365a166f40f0b7cb467c14594db0 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "163f365a166f40f0b7cb467c14594db0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57490e6474a34cee8c281b1654785252" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "163f365a166f40f0b7cb467c14594db0" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 39047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297854506d68483bb8b94da0b5518749" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39439 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[14621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:21.411991 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.403770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.13.195.65:39439)
0619 20:44:21.403855 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.403858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.403957 (+ 99us) server_negotiation.cc:407] Connection header received
0619 20:44:21.404042 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.404043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.404053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.404124 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:21.404762 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.405255 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.406026 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.406217 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.406323 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.408925 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:21.408941 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:21.408941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:21.408951 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:21.410685 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.411076 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.411080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.411081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.411161 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.411406 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.411407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.411407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.411560 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:21.411725 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.411765 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.411859 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[14621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[14621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:21.435587 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.428226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.13.195.65:39439)
0619 20:44:21.428296 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.428298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.428447 (+ 149us) server_negotiation.cc:407] Connection header received
0619 20:44:21.428576 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.428577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.428592 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.428655 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:21.429226 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.429751 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.430511 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.430714 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.430776 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.433041 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:21.433042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:21.433043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:21.433049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:21.434389 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.434785 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.434786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.434786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.434843 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.435042 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.435043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.435043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.435170 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:21.435255 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.435450 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.435519 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[14621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[14621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:21.459908 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:21.450848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.13.195.65:39439)
0619 20:44:21.450925 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:21.450928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:21.451040 (+ 112us) server_negotiation.cc:407] Connection header received
0619 20:44:21.451144 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:21.451146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:21.451164 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:21.451259 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:21.451960 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.452519 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.453411 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:21.453617 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:21.453715 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:21.457092 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:21.457094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:21.457094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:21.457103 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:21.458898 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.459244 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.459246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.459246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.459304 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:21.459501 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:21.459502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:21.459502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:21.459594 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:21.459663 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:21.459761 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 20:44:21.459841 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 20:44:21.460469 14774 tablet_service.cc:2908] Rejecting scan request for tablet fb1822d62d26419690619f7df3717878: Uninitialized: clean time has not yet been initialized
I20260619 20:44:21.461337 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 14730
I20260619 20:44:21.467006 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 14867
I20260619 20:44:21.472275 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 15002
I20260619 20:44:21.476853 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 14634
2026-06-19T20:44:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2325 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[15174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[15174](info): set up 2 sockets
Jun 19 20:44:21 dist-test-slave-n2td krb5kdc[15174](info): commencing operation
krb5kdc: starting...
W20260619 20:44:22.533056 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:22 dist-test-slave-n2td krb5kdc[15174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:22.651082 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42235
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:42235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:22.734908 15187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:22.735164 15187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:22.735205 15187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:22.736447 15187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:22.736521 15187 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:44:22.736545 15187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:22.736579 15187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:22.736613 15187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:22.736644 15187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:22.738121 15187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:42235
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:22.738412 15187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:22.738685 15187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:22.743763 15195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:22.743788 15192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:22.743973 15187 server_base.cc:1034] running on GCE node
W20260619 20:44:22.743852 15193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:22.744402 15187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:22.744762 15187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:22.745954 15187 hybrid_clock.cc:648] HybridClock initialized: now 1781901862745946 us; error 56 us; skew 500 ppm
Jun 19 20:44:22 dist-test-slave-n2td krb5kdc[15174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901862, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:22.753434 15187 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:22.753991 15187 webserver.cc:460] Webserver started at http://127.13.195.126:38415/ using document root <none> and password file <none>
I20260619 20:44:22.754264 15187 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:22.754350 15187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:22.754536 15187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:22.755702 15187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "7a9018e255594a6e957722ed88cb9709"
format_stamp: "Formatted at 2026-06-19 20:44:22 on dist-test-slave-n2td"
server_key: "37c401d9ebb2d95ed50afe3809d7119c"
server_key_iv: "74936173dc568c3920902213aa60fb53"
server_key_version: "encryptionkey@0"
I20260619 20:44:22.756099 15187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "7a9018e255594a6e957722ed88cb9709"
format_stamp: "Formatted at 2026-06-19 20:44:22 on dist-test-slave-n2td"
server_key: "37c401d9ebb2d95ed50afe3809d7119c"
server_key_iv: "74936173dc568c3920902213aa60fb53"
server_key_version: "encryptionkey@0"
I20260619 20:44:22.757699 15187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:44:22.758531 15202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:22.758719 15187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:22.758833 15187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "7a9018e255594a6e957722ed88cb9709"
format_stamp: "Formatted at 2026-06-19 20:44:22 on dist-test-slave-n2td"
server_key: "37c401d9ebb2d95ed50afe3809d7119c"
server_key_iv: "74936173dc568c3920902213aa60fb53"
server_key_version: "encryptionkey@0"
I20260619 20:44:22.758929 15187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:22.776816 15187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:22.779027 15187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:22.779156 15187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:22.784632 15187 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:42235
I20260619 20:44:22.784708 15264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:42235 every 8 connection(s)
I20260619 20:44:22.785104 15187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:22.785840 15265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:22.787200 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 15187
I20260619 20:44:22.787292 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:22.787492 14093 external_mini_cluster.cc:1428] Setting key 1dee2bf3c198f374ff20d41223fd3bb6
I20260619 20:44:22.788821 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709: Bootstrap starting.
I20260619 20:44:22.789520 15265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:22.789873 15265 log.cc:824] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:22.790717 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709: No bootstrap required, opened a new log
I20260619 20:44:22.793102 15265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9018e255594a6e957722ed88cb9709" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42235 } }
I20260619 20:44:22.793278 15265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:22.793334 15265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9018e255594a6e957722ed88cb9709, State: Initialized, Role: FOLLOWER
I20260619 20:44:22.793480 15265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9018e255594a6e957722ed88cb9709" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42235 } }
I20260619 20:44:22.793565 15265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:22.793612 15265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:22.793664 15265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:22.794209 15265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9018e255594a6e957722ed88cb9709" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42235 } }
I20260619 20:44:22.794337 15265 leader_election.cc:302] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a9018e255594a6e957722ed88cb9709; no voters:
I20260619 20:44:22.794579 15265 leader_election.cc:288] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:22.794608 15270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:22.794734 15270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [term 1 LEADER]: Becoming Leader. State: Replica: 7a9018e255594a6e957722ed88cb9709, State: Running, Role: LEADER
I20260619 20:44:22.794901 15270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9018e255594a6e957722ed88cb9709" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42235 } }
Jun 19 20:44:22 dist-test-slave-n2td krb5kdc[15174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:22.794909 15265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:22.795356 15272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a9018e255594a6e957722ed88cb9709. Latest consensus state: current_term: 1 leader_uuid: "7a9018e255594a6e957722ed88cb9709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9018e255594a6e957722ed88cb9709" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42235 } } }
I20260619 20:44:22.795455 15272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:22.795575 15271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a9018e255594a6e957722ed88cb9709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9018e255594a6e957722ed88cb9709" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42235 } } }
I20260619 20:44:22.795662 15271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:22.795955 15275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:22.796748 15275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:22.798549 15275 catalog_manager.cc:1347] Generated new cluster ID: ffca7a5de8114c2c8d8ab38d669bb292
I20260619 20:44:22.798607 15275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:22.799471 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:22.788090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.13.195.126:42235)
0619 20:44:22.788311 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:22.788322 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:22.788364 (+ 42us) server_negotiation.cc:407] Connection header received
0619 20:44:22.788916 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:22.788925 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:22.788989 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:22.789151 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:22.789768 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:22.791169 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:22.791951 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:22.792634 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:22.792755 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:22.795634 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:22.795638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:22.795639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:22.795650 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:22.797843 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:22.798308 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:22.798310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:22.798311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:22.798455 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:22.798678 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:22.798679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:22.798679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:22.798908 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:22.799064 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:22.799169 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 20:44:22.799304 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260619 20:44:22.821370 15275 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:22.821941 15275 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:22.827837 15275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709: Generated new TSK 0
I20260619 20:44:22.828078 15275 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:22.865115 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42235
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:22.964079 15292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:22.964284 15292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:22.964308 15292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:22.965428 15292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:22.965468 15292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:22.965489 15292 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:22.965507 15292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:22.966914 15292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:22.967099 15292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:22.967275 15292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:44:22.972319 15292 server_base.cc:1034] running on GCE node
W20260619 20:44:22.972293 15298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:22.972294 15301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:22.972383 15299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:22.972738 15292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:22.973016 15292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:22.974211 15292 hybrid_clock.cc:648] HybridClock initialized: now 1781901862974185 us; error 36 us; skew 500 ppm
Jun 19 20:44:22 dist-test-slave-n2td krb5kdc[15174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901862, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:22.976802 15292 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:22.977174 15292 webserver.cc:460] Webserver started at http://127.13.195.65:42985/ using document root <none> and password file <none>
I20260619 20:44:22.977352 15292 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:22.977392 15292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:22.977479 15292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:22.978564 15292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "1dfb245d1264418ab1083b5ac7e7b209"
format_stamp: "Formatted at 2026-06-19 20:44:22 on dist-test-slave-n2td"
server_key: "e63e7e8e3379c079a5c3a2f24bb6fb47"
server_key_iv: "fac28c2a13b8c977c49308c1b2fd222a"
server_key_version: "encryptionkey@0"
I20260619 20:44:22.978888 15292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "1dfb245d1264418ab1083b5ac7e7b209"
format_stamp: "Formatted at 2026-06-19 20:44:22 on dist-test-slave-n2td"
server_key: "e63e7e8e3379c079a5c3a2f24bb6fb47"
server_key_iv: "fac28c2a13b8c977c49308c1b2fd222a"
server_key_version: "encryptionkey@0"
I20260619 20:44:22.980113 15292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:22.980819 15309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:22.981009 15292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:22.981088 15292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "1dfb245d1264418ab1083b5ac7e7b209"
format_stamp: "Formatted at 2026-06-19 20:44:22 on dist-test-slave-n2td"
server_key: "e63e7e8e3379c079a5c3a2f24bb6fb47"
server_key_iv: "fac28c2a13b8c977c49308c1b2fd222a"
server_key_version: "encryptionkey@0"
I20260619 20:44:22.981141 15292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:22.991977 15292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:22.993899 15292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:22.994005 15292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:22.994493 15292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:22.994534 15292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:22.994581 15292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:22.994608 15292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:23.001803 15292 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:44355
I20260619 20:44:23.001873 15426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:44355 every 8 connection(s)
I20260619 20:44:23.002228 15292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:44:22 dist-test-slave-n2td krb5kdc[15174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901862, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:23.005609 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:22.995026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:51935 (local address 127.13.195.126:42235)
0619 20:44:22.995093 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:22.995096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:22.995384 (+ 288us) server_negotiation.cc:407] Connection header received
0619 20:44:22.995949 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:22.995951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:22.995966 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:22.996030 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:22.997049 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:22.997889 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:22.999581 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:22.999920 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:22.999974 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.002938 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:23.002939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:23.002939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:23.002969 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:23.004325 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.004744 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.004745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.004745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.004795 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.005059 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.005060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.005060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.005166 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:23.005215 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.005476 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.005543 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:44:23.005779 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:22.995017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42235 (local address 127.13.195.65:51935)
0619 20:44:22.995199 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:22.995238 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:22.995759 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:22.996168 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:22.996174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:22.996579 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:22.996943 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:22.996949 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:22.997975 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:22.997976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:22.999471 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:22.999474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.000008 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.000009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.000170 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.000922 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:23.000933 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:23.002847 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:23.004414 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.004421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.004422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.004673 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.004885 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.004886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.004886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.004998 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.005223 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:23.005227 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:23.005347 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.005435 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.005556 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":116,"threads_started":1}
I20260619 20:44:23.006559 15427 heartbeater.cc:347] Connected to a master server at 127.13.195.126:42235
I20260619 20:44:23.006677 15427 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:23.006875 15427 heartbeater.cc:511] Master 127.13.195.126:42235 requested a full tablet report, sending...
I20260619 20:44:23.007470 15218 ts_manager.cc:194] Registered new tserver with Master: 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355)
I20260619 20:44:23.008138 15218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:51935
I20260619 20:44:23.010818 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 15292
I20260619 20:44:23.010949 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:23.011183 14093 external_mini_cluster.cc:1428] Setting key cc1454a41953ea538fe988d8619cd16d
I20260619 20:44:23.014003 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.007625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:42011 (local address 127.13.195.126:42235)
0619 20:44:23.007663 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.007664 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.007803 (+ 139us) server_negotiation.cc:407] Connection header received
0619 20:44:23.007863 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.007864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.007874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.007947 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:23.008321 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.008984 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.009641 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.009832 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.009886 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.011079 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:23.011080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:23.011081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:23.011087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:23.012678 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.013118 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.013119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.013120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.013176 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.013413 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.013414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.013414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.013546 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:23.013638 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.013814 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.013906 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:23.014017 15433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.007561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42235 (local address 127.13.195.65:42011)
0619 20:44:23.007723 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.007732 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.007804 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.007954 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.007955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.008135 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:23.008210 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.008212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.009075 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.009076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.009571 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.009572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.009932 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.009933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.010002 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.010485 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:23.010491 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:23.010980 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:23.012771 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.012772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.012773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.013028 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.013251 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.013252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.013252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.013324 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.013644 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:23.013645 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:23.013709 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.013787 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.013908 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:23.037168 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42235
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:23.117887 15436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:23.118144 15436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:23.118183 15436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:23.119369 15436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:23.119452 15436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:23.119488 15436 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:23.119525 15436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:23.120952 15436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:23.121222 15436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:23.121459 15436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:23.126619 15441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:23.126648 15436 server_base.cc:1034] running on GCE node
W20260619 20:44:23.126725 15442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:23.126629 15445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:23.127131 15436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:23.127386 15436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:23.128559 15436 hybrid_clock.cc:648] HybridClock initialized: now 1781901863128550 us; error 34 us; skew 500 ppm
Jun 19 20:44:23 dist-test-slave-n2td krb5kdc[15174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901863, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:23.131229 15436 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:23.131752 15436 webserver.cc:460] Webserver started at http://127.13.195.66:36741/ using document root <none> and password file <none>
I20260619 20:44:23.132001 15436 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:23.132071 15436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:23.132218 15436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:23.133412 15436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "60b0d8d331334bf7ba22951fd8d2acaf"
format_stamp: "Formatted at 2026-06-19 20:44:23 on dist-test-slave-n2td"
server_key: "4211aad32e28f89e4d2268ef46829f41"
server_key_iv: "37e2347ba6963718b9f7917989b94a33"
server_key_version: "encryptionkey@0"
I20260619 20:44:23.133787 15436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "60b0d8d331334bf7ba22951fd8d2acaf"
format_stamp: "Formatted at 2026-06-19 20:44:23 on dist-test-slave-n2td"
server_key: "4211aad32e28f89e4d2268ef46829f41"
server_key_iv: "37e2347ba6963718b9f7917989b94a33"
server_key_version: "encryptionkey@0"
I20260619 20:44:23.135267 15436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:23.136111 15452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:23.136374 15436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:23.136498 15436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "60b0d8d331334bf7ba22951fd8d2acaf"
format_stamp: "Formatted at 2026-06-19 20:44:23 on dist-test-slave-n2td"
server_key: "4211aad32e28f89e4d2268ef46829f41"
server_key_iv: "37e2347ba6963718b9f7917989b94a33"
server_key_version: "encryptionkey@0"
I20260619 20:44:23.136571 15436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:23.157083 15436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:23.159198 15436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:23.159371 15436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:23.160024 15436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:23.160086 15436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:23.160130 15436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:23.160194 15436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:44:23 dist-test-slave-n2td krb5kdc[15174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901863, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:23.167637 15436 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:43775
I20260619 20:44:23.167725 15567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:43775 every 8 connection(s)
I20260619 20:44:23.168111 15436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:23.170396 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.160462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:40601 (local address 127.13.195.126:42235)
0619 20:44:23.160532 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.160535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.160724 (+ 189us) server_negotiation.cc:407] Connection header received
0619 20:44:23.161276 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.161277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.161298 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.161354 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:23.162239 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.163015 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.164148 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.164365 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.164455 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.167418 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:23.167420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:23.167421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:23.167430 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:23.169051 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.169509 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.169510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.169511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.169565 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.169834 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.169835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.169835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.169958 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:23.170021 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.170257 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.170329 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:44:23.170603 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.160438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42235 (local address 127.13.195.66:40601)
0619 20:44:23.160585 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.160632 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.161155 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.161426 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.161430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.161761 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:23.162120 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.162126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.163119 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.163120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.164036 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.164039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.164466 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.164468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.164610 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.165202 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:23.165213 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:23.167286 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:23.169125 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.169129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.169131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.169438 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.169631 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.169632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.169632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.169730 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.170022 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:23.170026 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:23.170143 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.170227 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.170343 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260619 20:44:23.171131 15568 heartbeater.cc:347] Connected to a master server at 127.13.195.126:42235
I20260619 20:44:23.171216 15568 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:23.171446 15568 heartbeater.cc:511] Master 127.13.195.126:42235 requested a full tablet report, sending...
I20260619 20:44:23.171821 15219 ts_manager.cc:194] Registered new tserver with Master: 60b0d8d331334bf7ba22951fd8d2acaf (127.13.195.66:43775)
I20260619 20:44:23.172094 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 15436
I20260619 20:44:23.172183 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:23.172358 14093 external_mini_cluster.cc:1428] Setting key 683b80f90402d2b4670842c56ca8b56b
I20260619 20:44:23.172348 15219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:40601
I20260619 20:44:23.178627 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.172473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:57199 (local address 127.13.195.126:42235)
0619 20:44:23.172517 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.172519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.172528 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:23.172549 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.172550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.172560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.172613 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:23.173049 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.173556 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.174639 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.174966 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.175042 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.176073 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:23.176074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:23.176075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:23.176081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:23.177382 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.177801 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.177802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.177802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.177888 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.178099 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.178100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.178100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.178227 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:23.178277 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.178505 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.178565 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:23.178741 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.172194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42235 (local address 127.13.195.66:57199)
0619 20:44:23.172306 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.172316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.172367 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.172620 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.172621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.172901 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:23.172981 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.172983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.173658 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.173660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.174562 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.174564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.175045 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.175046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.175103 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.175579 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:23.175585 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:23.175990 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:23.177491 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.177492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.177492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.177727 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.177958 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.177959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.177959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.178032 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.178300 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:23.178301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:23.178355 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.178467 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.178589 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:23.200619 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42235
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:23.285423 15577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:23.285662 15577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:23.285710 15577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:23.286988 15577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:23.287052 15577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:23.287089 15577 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:23.287144 15577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:23.288590 15577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:23.288856 15577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:23.289088 15577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:23.294232 15585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:23.294234 15583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:23.294260 15582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:23.294687 15577 server_base.cc:1034] running on GCE node
I20260619 20:44:23.294952 15577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:23.295217 15577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:23.296419 15577 hybrid_clock.cc:648] HybridClock initialized: now 1781901863296373 us; error 56 us; skew 500 ppm
Jun 19 20:44:23 dist-test-slave-n2td krb5kdc[15174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901863, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:23.299005 15577 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:23.299499 15577 webserver.cc:460] Webserver started at http://127.13.195.67:43311/ using document root <none> and password file <none>
I20260619 20:44:23.299937 15577 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:23.300053 15577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:23.300220 15577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:23.301352 15577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "35ee02e01aea4708ac2113afa8ec1545"
format_stamp: "Formatted at 2026-06-19 20:44:23 on dist-test-slave-n2td"
server_key: "f1e932ea9d3fefc954fd8df899903355"
server_key_iv: "2672756961d70e2f4eb6f6727624be4d"
server_key_version: "encryptionkey@0"
I20260619 20:44:23.301736 15577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "35ee02e01aea4708ac2113afa8ec1545"
format_stamp: "Formatted at 2026-06-19 20:44:23 on dist-test-slave-n2td"
server_key: "f1e932ea9d3fefc954fd8df899903355"
server_key_iv: "2672756961d70e2f4eb6f6727624be4d"
server_key_version: "encryptionkey@0"
I20260619 20:44:23.303161 15577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:23.303946 15592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:23.304124 15577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:23.304229 15577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "35ee02e01aea4708ac2113afa8ec1545"
format_stamp: "Formatted at 2026-06-19 20:44:23 on dist-test-slave-n2td"
server_key: "f1e932ea9d3fefc954fd8df899903355"
server_key_iv: "2672756961d70e2f4eb6f6727624be4d"
server_key_version: "encryptionkey@0"
I20260619 20:44:23.304303 15577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:23.329032 15577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:23.331514 15577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:23.331693 15577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:23.332273 15577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:23.332340 15577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:23.332379 15577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:23.332409 15577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:44:23 dist-test-slave-n2td krb5kdc[15174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901863, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:23.339875 15577 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:35377
I20260619 20:44:23.339958 15710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:35377 every 8 connection(s)
I20260619 20:44:23.340317 15577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:23.342881 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.332859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:39259 (local address 127.13.195.126:42235)
0619 20:44:23.332939 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.332941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.333315 (+ 374us) server_negotiation.cc:407] Connection header received
0619 20:44:23.333988 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.333989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.334004 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.334075 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:23.334941 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.335475 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.336618 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.336840 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.336934 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.339785 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:23.339787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:23.339787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:23.339796 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:23.341444 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.341862 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.341863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.341864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.341921 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.342193 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.342194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.342194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.342320 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:23.342401 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.342703 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.342781 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:44:23.343061 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.332828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42235 (local address 127.13.195.67:39259)
0619 20:44:23.333167 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.333210 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.333829 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.334197 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.334202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.334615 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:23.334824 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.334829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.335580 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.335581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.336498 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.336500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.336958 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.336959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.337093 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.337692 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:23.337703 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:23.339698 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:23.341540 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.341547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.341549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.341767 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.341994 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.341995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.341995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.342108 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.342391 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:23.342397 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:23.342562 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.342669 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.342828 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":103,"threads_started":1}
I20260619 20:44:23.343561 15711 heartbeater.cc:347] Connected to a master server at 127.13.195.126:42235
I20260619 20:44:23.343641 15711 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:23.343833 15711 heartbeater.cc:511] Master 127.13.195.126:42235 requested a full tablet report, sending...
I20260619 20:44:23.344199 15218 ts_manager.cc:194] Registered new tserver with Master: 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67:35377)
I20260619 20:44:23.344687 15218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:39259
I20260619 20:44:23.345916 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 15577
I20260619 20:44:23.346006 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:23.346159 14093 external_mini_cluster.cc:1428] Setting key dbc318c0b715c5e37ed7a7d2b3ba197f
I20260619 20:44:23.347095 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:44:23.350780 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.344604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42235 (local address 127.13.195.67:43289)
0619 20:44:23.344801 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.344823 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.344881 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.345009 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.345009 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.345208 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:23.345279 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.345280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.345912 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.345912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.346725 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.346727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.347112 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.347113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.347187 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.347629 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:23.347633 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:23.347893 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:23.349648 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.349648 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.349649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.349858 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.350064 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:23.350064 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:23.350065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:23.350133 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:23.350508 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:23.350510 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:23.350559 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.350623 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.350704 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
I20260619 20:44:23.350881 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.344710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:43289 (local address 127.13.195.126:42235)
0619 20:44:23.344752 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.344754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.344870 (+ 116us) server_negotiation.cc:407] Connection header received
0619 20:44:23.344939 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.344940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.344950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.344999 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:23.345353 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.345810 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.346804 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.347022 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.347095 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.347980 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:23.347981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:23.347982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:23.347989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:23.349548 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.349942 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.349943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.349944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.350002 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.350194 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.350195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.350196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.350401 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:23.350495 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.350669 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.350750 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:44:23 dist-test-slave-n2td krb5kdc[15174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901863, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:44:23 dist-test-slave-n2td krb5kdc[15174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901863, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:23.370755 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.363510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.13.195.126:42235)
0619 20:44:23.363572 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.363574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.363588 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:44:23.363654 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.363655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.363664 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.363715 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:23.364270 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.364765 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.365506 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.365690 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.365757 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.368100 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:23.368101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:23.368102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:23.368109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:23.369481 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.369845 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.369846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.369846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.369912 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:23.370151 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:23.370152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:23.370153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:23.370286 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:23.370392 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.370554 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.370640 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:44:23.372711 15218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:23.374469 15218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:23.381623 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.377186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.13.195.67:35377)
0619 20:44:23.377360 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.377365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.377761 (+ 396us) server_negotiation.cc:407] Connection header received
0619 20:44:23.377806 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.377808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.377823 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.377901 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:23.378090 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.379140 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.380426 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.381142 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.381221 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.381279 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.381476 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.381520 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260619 20:44:23.381618 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.376949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:35377 (local address 127.0.0.1:59680)
0619 20:44:23.377643 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.377653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.377701 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.377967 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.377967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.377969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:23.378004 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.378005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.379230 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.379231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.380318 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.380320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.381235 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.381236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.381298 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.381313 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.381451 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.381518 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260619 20:44:23.381870 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.376925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:44355 (local address 127.0.0.1:57262)
0619 20:44:23.377194 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.377236 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.377329 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.377576 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.377579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.377581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:23.377686 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.377688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.379071 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.379072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.380553 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.380557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.381421 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.381422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.381486 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.381489 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.381544 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.381580 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":67,"thread_start_us":56,"threads_started":1}
I20260619 20:44:23.381870 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.377015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.13.195.65:44355)
0619 20:44:23.377186 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.377191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.377430 (+ 239us) server_negotiation.cc:407] Connection header received
0619 20:44:23.377475 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.377477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.377490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.377571 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:23.377810 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.378987 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.380650 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.381334 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.381402 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.381467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.381694 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.381748 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260619 20:44:23.382777 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.376948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:43775 (local address 127.0.0.1:47976)
0619 20:44:23.377439 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.377450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.377548 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.377804 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.377805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.377807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:23.377884 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.377886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.378920 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.378921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.380561 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.380562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.382541 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.382542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.382601 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.382605 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.382663 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.382703 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":139,"thread_start_us":101,"threads_started":1}
I20260619 20:44:23.382864 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.377186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.13.195.66:43775)
0619 20:44:23.377408 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.377414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.377538 (+ 124us) server_negotiation.cc:407] Connection header received
0619 20:44:23.377625 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.377627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.377644 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.377731 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:23.377968 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.378817 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.381502 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.382350 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.382450 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.382507 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.382703 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.382748 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":110,"threads_started":1}
I20260619 20:44:23.382799 15644 tablet_service.cc:1467] Processing CreateTablet for tablet 2c081fe3501849f39647418c49e0634a (DEFAULT_TABLE table=test-table [id=c938ca01ffee4dbf898756520ad07ea5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:23.382879 15361 tablet_service.cc:1467] Processing CreateTablet for tablet 2c081fe3501849f39647418c49e0634a (DEFAULT_TABLE table=test-table [id=c938ca01ffee4dbf898756520ad07ea5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:23.383195 15644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c081fe3501849f39647418c49e0634a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:23.383282 15361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c081fe3501849f39647418c49e0634a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:23.385972 15734 tablet_bootstrap.cc:492] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Bootstrap starting.
I20260619 20:44:23.386952 15502 tablet_service.cc:1467] Processing CreateTablet for tablet 2c081fe3501849f39647418c49e0634a (DEFAULT_TABLE table=test-table [id=c938ca01ffee4dbf898756520ad07ea5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:23.387255 15502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c081fe3501849f39647418c49e0634a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:23.387274 15734 tablet_bootstrap.cc:654] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:23.387598 15734 log.cc:824] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:23.387933 15735 tablet_bootstrap.cc:492] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Bootstrap starting.
I20260619 20:44:23.388398 15734 tablet_bootstrap.cc:492] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: No bootstrap required, opened a new log
I20260619 20:44:23.388468 15734 ts_tablet_manager.cc:1397] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:44:23.388615 15735 tablet_bootstrap.cc:654] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:23.388953 15735 log.cc:824] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:23.389635 15735 tablet_bootstrap.cc:492] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: No bootstrap required, opened a new log
I20260619 20:44:23.389680 15735 ts_tablet_manager.cc:1397] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:23.390668 15734 raft_consensus.cc:348] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.390848 15738 tablet_bootstrap.cc:492] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Bootstrap starting.
I20260619 20:44:23.390956 15734 raft_consensus.cc:374] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:23.391037 15734 raft_consensus.cc:729] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ee02e01aea4708ac2113afa8ec1545, State: Initialized, Role: FOLLOWER
I20260619 20:44:23.391170 15734 consensus_queue.cc:260] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.391469 15711 heartbeater.cc:503] Master 127.13.195.126:42235 was elected leader, sending a full tablet report...
I20260619 20:44:23.391458 15734 ts_tablet_manager.cc:1428] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:44:23.391758 15738 tablet_bootstrap.cc:654] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:23.392398 15738 log.cc:824] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:23.392453 15735 raft_consensus.cc:348] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.392627 15735 raft_consensus.cc:374] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:23.392678 15735 raft_consensus.cc:729] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfb245d1264418ab1083b5ac7e7b209, State: Initialized, Role: FOLLOWER
I20260619 20:44:23.392825 15735 consensus_queue.cc:260] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.393173 15735 ts_tablet_manager.cc:1428] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 20:44:23.393301 15427 heartbeater.cc:503] Master 127.13.195.126:42235 was elected leader, sending a full tablet report...
I20260619 20:44:23.393563 15738 tablet_bootstrap.cc:492] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: No bootstrap required, opened a new log
I20260619 20:44:23.393649 15738 ts_tablet_manager.cc:1397] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:44:23.395876 15738 raft_consensus.cc:348] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.396081 15738 raft_consensus.cc:374] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:23.396132 15738 raft_consensus.cc:729] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b0d8d331334bf7ba22951fd8d2acaf, State: Initialized, Role: FOLLOWER
I20260619 20:44:23.396260 15738 consensus_queue.cc:260] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.396553 15738 ts_tablet_manager.cc:1428] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:44:23.396597 15568 heartbeater.cc:503] Master 127.13.195.126:42235 was elected leader, sending a full tablet report...
W20260619 20:44:23.418795 15569 tablet.cc:2369] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:23.502807 15739 raft_consensus.cc:482] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:23.502991 15739 raft_consensus.cc:504] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
W20260619 20:44:23.503084 15428 tablet.cc:2369] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:23.503417 15739 leader_election.cc:288] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60b0d8d331334bf7ba22951fd8d2acaf (127.13.195.66:43775), 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355)
I20260619 20:44:23.507714 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.503686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:39469 (local address 127.13.195.66:43775)
0619 20:44:23.503747 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.503750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.503840 (+ 90us) server_negotiation.cc:407] Connection header received
0619 20:44:23.503906 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.503907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.503918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.504011 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:23.504270 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.505020 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.506719 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.507329 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.507389 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.507398 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.507603 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.507643 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:44:23.507714 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.503713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:43971 (local address 127.13.195.65:44355)
0619 20:44:23.503756 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.503757 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.504083 (+ 326us) server_negotiation.cc:407] Connection header received
0619 20:44:23.504189 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.504190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.504197 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.504233 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:23.504404 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.505083 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.506732 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.507307 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.507395 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.507404 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.507604 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.507643 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:44:23.507741 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.503686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:43775 (local address 127.13.195.67:39469)
0619 20:44:23.503755 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.503767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.503851 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.504028 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.504029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.504031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:23.504111 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.504114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.505106 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.505106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.506619 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.506622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.507409 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.507410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.507520 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.507522 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.507587 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.507638 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":91}
I20260619 20:44:23.507750 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.503687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:44355 (local address 127.13.195.67:43971)
0619 20:44:23.504024 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.504034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.504117 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.504255 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.504255 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.504257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:23.504329 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.504331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.505155 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.505156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.506619 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.506622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.507410 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.507411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.507517 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.507522 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.507587 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.507637 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":122,"thread_start_us":142,"threads_started":1}
I20260619 20:44:23.508230 15522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c081fe3501849f39647418c49e0634a" candidate_uuid: "35ee02e01aea4708ac2113afa8ec1545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" is_pre_election: true
I20260619 20:44:23.508409 15522 raft_consensus.cc:2454] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35ee02e01aea4708ac2113afa8ec1545 in term 0.
I20260619 20:44:23.508406 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c081fe3501849f39647418c49e0634a" candidate_uuid: "35ee02e01aea4708ac2113afa8ec1545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfb245d1264418ab1083b5ac7e7b209" is_pre_election: true
I20260619 20:44:23.508534 15381 raft_consensus.cc:2454] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35ee02e01aea4708ac2113afa8ec1545 in term 0.
I20260619 20:44:23.508708 15593 leader_election.cc:302] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35ee02e01aea4708ac2113afa8ec1545, 60b0d8d331334bf7ba22951fd8d2acaf; no voters:
I20260619 20:44:23.508879 15739 raft_consensus.cc:2781] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:23.508963 15739 raft_consensus.cc:482] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:23.509001 15739 raft_consensus.cc:3037] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:23.510028 15739 raft_consensus.cc:504] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.510188 15739 leader_election.cc:288] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [CANDIDATE]: Term 1 election: Requested vote from peers 60b0d8d331334bf7ba22951fd8d2acaf (127.13.195.66:43775), 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355)
I20260619 20:44:23.510326 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c081fe3501849f39647418c49e0634a" candidate_uuid: "35ee02e01aea4708ac2113afa8ec1545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfb245d1264418ab1083b5ac7e7b209"
I20260619 20:44:23.510361 15522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c081fe3501849f39647418c49e0634a" candidate_uuid: "35ee02e01aea4708ac2113afa8ec1545" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60b0d8d331334bf7ba22951fd8d2acaf"
I20260619 20:44:23.510411 15381 raft_consensus.cc:3037] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:23.510462 15522 raft_consensus.cc:3037] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:23.511349 15381 raft_consensus.cc:2454] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35ee02e01aea4708ac2113afa8ec1545 in term 1.
I20260619 20:44:23.511505 15522 raft_consensus.cc:2454] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35ee02e01aea4708ac2113afa8ec1545 in term 1.
I20260619 20:44:23.511569 15594 leader_election.cc:302] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dfb245d1264418ab1083b5ac7e7b209, 35ee02e01aea4708ac2113afa8ec1545; no voters:
I20260619 20:44:23.511664 15739 raft_consensus.cc:2781] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:23.511845 15739 raft_consensus.cc:686] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 1 LEADER]: Becoming Leader. State: Replica: 35ee02e01aea4708ac2113afa8ec1545, State: Running, Role: LEADER
I20260619 20:44:23.511965 15739 consensus_queue.cc:237] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.512840 15218 catalog_manager.cc:5495] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "35ee02e01aea4708ac2113afa8ec1545" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } health_report { overall_health: HEALTHY } } }
I20260619 20:44:23.528002 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.525251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59682 (local address 127.13.195.67:35377)
0619 20:44:23.525305 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.525307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.525317 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:23.525422 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.525423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.525440 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.525528 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:23.525695 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.526171 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.527087 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.527278 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.527394 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.527616 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:23.527725 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:23.527815 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.527867 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.527887 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:44:23.530028 15381 raft_consensus.cc:1261] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Refusing update from remote peer 35ee02e01aea4708ac2113afa8ec1545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:23.530028 15522 raft_consensus.cc:1261] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Refusing update from remote peer 35ee02e01aea4708ac2113afa8ec1545: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:23.530406 15739 consensus_queue.cc:1035] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:23.530591 15745 consensus_queue.cc:1035] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:23.533623 15750 mvcc.cc:205] Tried to move back new op lower bound from 7298670033015562240 to 7298670032945664000. Current Snapshot: MvccSnapshot[applied={T|T < 7298670033015562240}]
I20260619 20:44:23.536396 15219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:23.536496 15219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:23.537271 15219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709: Sending DeleteTablet for 3 replicas of tablet 2c081fe3501849f39647418c49e0634a
I20260619 20:44:23.537640 15361 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c081fe3501849f39647418c49e0634a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:23 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:57262
I20260619 20:44:23.537698 15644 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c081fe3501849f39647418c49e0634a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:23 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:59680
I20260619 20:44:23.537840 15502 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c081fe3501849f39647418c49e0634a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:23 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:47976
I20260619 20:44:23.537892 15219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:23.538049 15219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:23.538113 15759 tablet_replica.cc:330] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: stopping tablet replica
I20260619 20:44:23.538250 15759 raft_consensus.cc:2229] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:23.538462 15759 raft_consensus.cc:2258] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:23.538662 15758 tablet_replica.cc:330] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: stopping tablet replica
I20260619 20:44:23.538770 15758 raft_consensus.cc:2229] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:23.538942 15758 raft_consensus.cc:2258] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:23.539110 15759 ts_tablet_manager.cc:1905] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:23.539414 15758 ts_tablet_manager.cc:1905] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:23.539887 15361 tablet_service.cc:1467] Processing CreateTablet for tablet 66c9d479cc254567b3ffd2b49eba5bf8 (DEFAULT_TABLE table=test-table [id=a55aeb99e5834e19b2b96f80f1ca329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:23.539929 15644 tablet_service.cc:1467] Processing CreateTablet for tablet 66c9d479cc254567b3ffd2b49eba5bf8 (DEFAULT_TABLE table=test-table [id=a55aeb99e5834e19b2b96f80f1ca329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:23.540015 15361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66c9d479cc254567b3ffd2b49eba5bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:23.540033 15644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66c9d479cc254567b3ffd2b49eba5bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:23.540647 15760 tablet_replica.cc:330] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: stopping tablet replica
I20260619 20:44:23.541034 15760 raft_consensus.cc:2229] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:23.541082 15759 ts_tablet_manager.cc:1918] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:23.541139 15759 log.cc:1196] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/2c081fe3501849f39647418c49e0634a
I20260619 20:44:23.541162 15760 raft_consensus.cc:2258] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:23.541386 15759 ts_tablet_manager.cc:1939] T 2c081fe3501849f39647418c49e0634a P 35ee02e01aea4708ac2113afa8ec1545: Deleting consensus metadata
I20260619 20:44:23.541523 15734 tablet_bootstrap.cc:492] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Bootstrap starting.
I20260619 20:44:23.541707 15760 ts_tablet_manager.cc:1905] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:23.541935 15734 tablet_bootstrap.cc:654] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:23.542899 15502 tablet_service.cc:1467] Processing CreateTablet for tablet 66c9d479cc254567b3ffd2b49eba5bf8 (DEFAULT_TABLE table=test-table [id=a55aeb99e5834e19b2b96f80f1ca329a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:23.542909 15206 catalog_manager.cc:4841] TS 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67:35377): tablet 2c081fe3501849f39647418c49e0634a (table test-table [id=c938ca01ffee4dbf898756520ad07ea5]) successfully deleted
I20260619 20:44:23.543004 15502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66c9d479cc254567b3ffd2b49eba5bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:23.543116 15760 ts_tablet_manager.cc:1918] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:23.543172 15760 log.cc:1196] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/2c081fe3501849f39647418c49e0634a
I20260619 20:44:23.543450 15760 ts_tablet_manager.cc:1939] T 2c081fe3501849f39647418c49e0634a P 60b0d8d331334bf7ba22951fd8d2acaf: Deleting consensus metadata
I20260619 20:44:23.543908 15203 catalog_manager.cc:4841] TS 60b0d8d331334bf7ba22951fd8d2acaf (127.13.195.66:43775): tablet 2c081fe3501849f39647418c49e0634a (table test-table [id=c938ca01ffee4dbf898756520ad07ea5]) successfully deleted
I20260619 20:44:23.544216 15738 tablet_bootstrap.cc:492] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: Bootstrap starting.
I20260619 20:44:23.544625 15738 tablet_bootstrap.cc:654] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:23.545964 15758 ts_tablet_manager.cc:1918] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:23.546036 15758 log.cc:1196] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/2c081fe3501849f39647418c49e0634a
I20260619 20:44:23.546041 15735 tablet_bootstrap.cc:492] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: Bootstrap starting.
I20260619 20:44:23.546293 15738 tablet_bootstrap.cc:492] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: No bootstrap required, opened a new log
I20260619 20:44:23.546312 15758 ts_tablet_manager.cc:1939] T 2c081fe3501849f39647418c49e0634a P 1dfb245d1264418ab1083b5ac7e7b209: Deleting consensus metadata
I20260619 20:44:23.546334 15738 ts_tablet_manager.cc:1397] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:23.546622 15738 raft_consensus.cc:348] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.546705 15738 raft_consensus.cc:374] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:23.546731 15738 raft_consensus.cc:729] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b0d8d331334bf7ba22951fd8d2acaf, State: Initialized, Role: FOLLOWER
I20260619 20:44:23.546792 15735 tablet_bootstrap.cc:654] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:23.546774 15738 consensus_queue.cc:260] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.546911 15738 ts_tablet_manager.cc:1428] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:23.547106 15204 catalog_manager.cc:4841] TS 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355): tablet 2c081fe3501849f39647418c49e0634a (table test-table [id=c938ca01ffee4dbf898756520ad07ea5]) successfully deleted
I20260619 20:44:23.547449 15734 tablet_bootstrap.cc:492] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: No bootstrap required, opened a new log
I20260619 20:44:23.547510 15734 ts_tablet_manager.cc:1397] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260619 20:44:23.547600 15735 tablet_bootstrap.cc:492] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: No bootstrap required, opened a new log
I20260619 20:44:23.547640 15735 ts_tablet_manager.cc:1397] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:23.547757 15735 raft_consensus.cc:348] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.547803 15735 raft_consensus.cc:374] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:23.547755 15734 raft_consensus.cc:348] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.547816 15735 raft_consensus.cc:729] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfb245d1264418ab1083b5ac7e7b209, State: Initialized, Role: FOLLOWER
I20260619 20:44:23.547824 15734 raft_consensus.cc:374] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:23.547857 15734 raft_consensus.cc:729] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ee02e01aea4708ac2113afa8ec1545, State: Initialized, Role: FOLLOWER
I20260619 20:44:23.547881 15735 consensus_queue.cc:260] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.547914 15734 consensus_queue.cc:260] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.547993 15734 ts_tablet_manager.cc:1428] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:23.548004 15735 ts_tablet_manager.cc:1428] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 20:44:23.591151 15712 tablet.cc:2369] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:23.962284 15743 raft_consensus.cc:482] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:23.962453 15743 raft_consensus.cc:504] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.962841 15743 leader_election.cc:288] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355), 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67:35377)
I20260619 20:44:23.966944 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.963139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:42921 (local address 127.13.195.67:35377)
0619 20:44:23.963187 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.963190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.963285 (+ 95us) server_negotiation.cc:407] Connection header received
0619 20:44:23.963349 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.963350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.963362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.963426 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:23.963613 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.964357 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.965961 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.966539 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.966616 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.966624 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.966830 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.966869 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:44:23.966944 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.963133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:59203 (local address 127.13.195.65:44355)
0619 20:44:23.963192 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:23.963194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:23.963285 (+ 91us) server_negotiation.cc:407] Connection header received
0619 20:44:23.963338 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:23.963339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:23.963349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:23.963471 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:23.963610 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.964356 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.965961 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.966538 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.966616 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.966624 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:23.966832 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 20:44:23.966869 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:44:23.966981 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.963053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:35377 (local address 127.13.195.66:42921)
0619 20:44:23.963195 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.963208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.963280 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.963430 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.963431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.963433 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:23.963533 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.963536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.964474 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.964475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.965846 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.965849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.966641 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.966643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.966740 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.966748 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.966810 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.966874 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":85,"thread_start_us":76,"threads_started":1}
I20260619 20:44:23.966984 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:23.963054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:44355 (local address 127.13.195.66:59203)
0619 20:44:23.963195 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:23.963223 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:23.963280 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:23.963485 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:23.963486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:23.963488 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:23.963533 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.963536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.964474 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.964475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.965846 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:23.965849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:23.966643 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:23.966643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:23.966740 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:23.966749 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:44:23.966810 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:44:23.966874 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":104,"thread_start_us":72,"threads_started":1}
I20260619 20:44:23.967401 15665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c9d479cc254567b3ffd2b49eba5bf8" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35ee02e01aea4708ac2113afa8ec1545" is_pre_election: true
I20260619 20:44:23.967442 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c9d479cc254567b3ffd2b49eba5bf8" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfb245d1264418ab1083b5ac7e7b209" is_pre_election: true
I20260619 20:44:23.967528 15381 raft_consensus.cc:2454] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 0.
I20260619 20:44:23.967558 15665 raft_consensus.cc:2454] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 0.
I20260619 20:44:23.967700 15454 leader_election.cc:302] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dfb245d1264418ab1083b5ac7e7b209, 60b0d8d331334bf7ba22951fd8d2acaf; no voters:
I20260619 20:44:23.967876 15743 raft_consensus.cc:2781] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:23.967922 15743 raft_consensus.cc:482] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:23.967937 15743 raft_consensus.cc:3037] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:23.968691 15743 raft_consensus.cc:504] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.968844 15743 leader_election.cc:288] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 election: Requested vote from peers 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355), 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67:35377)
I20260619 20:44:23.969020 15665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c9d479cc254567b3ffd2b49eba5bf8" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35ee02e01aea4708ac2113afa8ec1545"
I20260619 20:44:23.969103 15665 raft_consensus.cc:3037] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:23.969326 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c9d479cc254567b3ffd2b49eba5bf8" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfb245d1264418ab1083b5ac7e7b209"
I20260619 20:44:23.969405 15381 raft_consensus.cc:3037] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:23.969991 15665 raft_consensus.cc:2454] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 1.
I20260619 20:44:23.970176 15456 leader_election.cc:302] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35ee02e01aea4708ac2113afa8ec1545, 60b0d8d331334bf7ba22951fd8d2acaf; no voters:
I20260619 20:44:23.970252 15381 raft_consensus.cc:2454] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 1.
I20260619 20:44:23.970283 15743 raft_consensus.cc:2781] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:23.970470 15743 raft_consensus.cc:686] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 LEADER]: Becoming Leader. State: Replica: 60b0d8d331334bf7ba22951fd8d2acaf, State: Running, Role: LEADER
I20260619 20:44:23.970546 15743 consensus_queue.cc:237] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:23.971256 15219 catalog_manager.cc:5495] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf reported cstate change: term changed from 0 to 1, leader changed from <none> to 60b0d8d331334bf7ba22951fd8d2acaf (127.13.195.66). New cstate: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:44:24 dist-test-slave-n2td krb5kdc[15174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901862, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:24.011291 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.003275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42235 (local address 127.0.0.1:48186)
0619 20:44:24.003483 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:24.003496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:24.003581 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:24.003856 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:24.003857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:24.004289 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:24.004348 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:24.004350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.005057 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.005058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:24.005607 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:24.005609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.005959 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.005960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:24.006002 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.006580 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:24.006584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:24.008318 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:24.010075 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:24.010084 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:24.010087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:24.010278 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:24.010549 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:24.010550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:24.010550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:24.010609 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:24.010868 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:24.010875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:24.010938 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 20:44:24.011011 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:44:24.011166 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260619 20:44:24.011313 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.003305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.13.195.126:42235)
0619 20:44:24.003510 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:24.003512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:24.003649 (+ 137us) server_negotiation.cc:407] Connection header received
0619 20:44:24.003685 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:24.003686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:24.003701 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:24.003817 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:24.004459 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.004963 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.005688 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.005894 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.005949 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.008420 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:24.008421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:24.008422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:24.008433 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:24.009974 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:24.010356 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:24.010357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:24.010357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:24.010466 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:24.010684 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:24.010685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:24.010686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:24.010803 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:24.010876 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:24.011045 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 20:44:24.011222 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":164,"thread_start_us":140,"threads_started":1}
I20260619 20:44:24.013497 15219 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:48186:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:44:24.013698 15219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:24.015201 15502 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca379c2945345abb68537c43ae48144 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a71141fd4365418bb996cc673a0eebb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:24.015260 15361 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca379c2945345abb68537c43ae48144 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a71141fd4365418bb996cc673a0eebb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:24.015244 15644 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca379c2945345abb68537c43ae48144 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a71141fd4365418bb996cc673a0eebb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:24.015372 15361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca379c2945345abb68537c43ae48144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:24.015373 15502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca379c2945345abb68537c43ae48144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:24.015403 15644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca379c2945345abb68537c43ae48144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:24.016762 15734 tablet_bootstrap.cc:492] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: Bootstrap starting.
I20260619 20:44:24.016896 15735 tablet_bootstrap.cc:492] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: Bootstrap starting.
I20260619 20:44:24.016917 15738 tablet_bootstrap.cc:492] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: Bootstrap starting.
I20260619 20:44:24.017220 15734 tablet_bootstrap.cc:654] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:24.017314 15735 tablet_bootstrap.cc:654] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:24.017517 15738 tablet_bootstrap.cc:654] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:24.017747 15734 tablet_bootstrap.cc:492] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: No bootstrap required, opened a new log
I20260619 20:44:24.017802 15734 ts_tablet_manager.cc:1397] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:24.017949 15734 raft_consensus.cc:348] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.018051 15734 raft_consensus.cc:374] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:24.018074 15734 raft_consensus.cc:729] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ee02e01aea4708ac2113afa8ec1545, State: Initialized, Role: FOLLOWER
I20260619 20:44:24.018131 15735 tablet_bootstrap.cc:492] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: No bootstrap required, opened a new log
I20260619 20:44:24.018114 15734 consensus_queue.cc:260] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.018173 15735 ts_tablet_manager.cc:1397] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:24.018208 15734 ts_tablet_manager.cc:1428] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:24.018337 15738 tablet_bootstrap.cc:492] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: No bootstrap required, opened a new log
I20260619 20:44:24.018337 15735 raft_consensus.cc:348] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.018324 15755 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.018400 15735 raft_consensus.cc:374] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:24.018409 15755 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:24.018468 15735 raft_consensus.cc:729] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfb245d1264418ab1083b5ac7e7b209, State: Initialized, Role: FOLLOWER
I20260619 20:44:24.018517 15735 consensus_queue.cc:260] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.018623 15735 ts_tablet_manager.cc:1428] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:24.018378 15738 ts_tablet_manager.cc:1397] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:24.018692 15742 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.018819 15742 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:24.018931 15738 raft_consensus.cc:348] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.019022 15738 raft_consensus.cc:374] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:24.019057 15738 raft_consensus.cc:729] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b0d8d331334bf7ba22951fd8d2acaf, State: Initialized, Role: FOLLOWER
I20260619 20:44:24.019109 15738 consensus_queue.cc:260] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.019215 15738 ts_tablet_manager.cc:1428] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:24.019260 15743 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.019349 15743 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:24.046717 15743 raft_consensus.cc:482] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:24.046846 15743 raft_consensus.cc:504] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.047060 15743 leader_election.cc:288] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355), 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67:35377)
I20260619 20:44:24.047281 15665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca379c2945345abb68537c43ae48144" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35ee02e01aea4708ac2113afa8ec1545" is_pre_election: true
I20260619 20:44:24.047281 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca379c2945345abb68537c43ae48144" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfb245d1264418ab1083b5ac7e7b209" is_pre_election: true
I20260619 20:44:24.047399 15381 raft_consensus.cc:2454] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 0.
I20260619 20:44:24.047399 15665 raft_consensus.cc:2454] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 0.
I20260619 20:44:24.047552 15456 leader_election.cc:302] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35ee02e01aea4708ac2113afa8ec1545, 60b0d8d331334bf7ba22951fd8d2acaf; no voters:
I20260619 20:44:24.047674 15743 raft_consensus.cc:2781] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:24.047729 15743 raft_consensus.cc:482] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:24.047748 15743 raft_consensus.cc:3037] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:24.048396 15743 raft_consensus.cc:504] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.048532 15743 leader_election.cc:288] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 election: Requested vote from peers 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355), 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67:35377)
I20260619 20:44:24.048655 15381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca379c2945345abb68537c43ae48144" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfb245d1264418ab1083b5ac7e7b209"
I20260619 20:44:24.048758 15381 raft_consensus.cc:3037] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:24.048707 15665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca379c2945345abb68537c43ae48144" candidate_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35ee02e01aea4708ac2113afa8ec1545"
I20260619 20:44:24.048801 15665 raft_consensus.cc:3037] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:24.049372 15665 raft_consensus.cc:2454] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 1.
I20260619 20:44:24.049371 15381 raft_consensus.cc:2454] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60b0d8d331334bf7ba22951fd8d2acaf in term 1.
I20260619 20:44:24.049562 15454 leader_election.cc:302] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dfb245d1264418ab1083b5ac7e7b209, 60b0d8d331334bf7ba22951fd8d2acaf; no voters:
I20260619 20:44:24.049664 15743 raft_consensus.cc:2781] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:24.049746 15743 raft_consensus.cc:686] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 LEADER]: Becoming Leader. State: Replica: 60b0d8d331334bf7ba22951fd8d2acaf, State: Running, Role: LEADER
I20260619 20:44:24.049800 15743 consensus_queue.cc:237] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } }
I20260619 20:44:24.050102 15766 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: TxnStatusTablet state changed. Reason: New leader 60b0d8d331334bf7ba22951fd8d2acaf. Latest consensus state: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.050199 15766 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:24.050339 15779 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:44:24.050499 15779 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:44:24.050405 15218 catalog_manager.cc:5495] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf reported cstate change: term changed from 0 to 1, leader changed from <none> to 60b0d8d331334bf7ba22951fd8d2acaf (127.13.195.66). New cstate: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:24.059832 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.056828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:43775 (local address 127.0.0.1:47982)
0619 20:44:24.056926 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:24.056939 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:24.056998 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:24.057225 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:24.057226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:24.057227 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:24.057278 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:24.057280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.058076 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.058078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:24.058760 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:24.058763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.059216 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.059217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:24.059283 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.059298 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:24.059626 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:24.059636 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 20:44:24.059694 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:44:24.059748 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 20:44:24.059875 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.056944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47982 (local address 127.13.195.66:43775)
0619 20:44:24.057055 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:24.057058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:24.057066 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:24.057106 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:24.057107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:24.057125 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:24.057219 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:24.057341 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.057986 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.058852 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.059129 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.059205 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.059410 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:24.059554 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:24.059643 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:24.059699 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 20:44:24.059721 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":50,"threads_started":1}
I20260619 20:44:24.061201 15381 raft_consensus.cc:1261] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Refusing update from remote peer 60b0d8d331334bf7ba22951fd8d2acaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:24.061251 15665 raft_consensus.cc:1261] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Refusing update from remote peer 60b0d8d331334bf7ba22951fd8d2acaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:24.061476 15766 consensus_queue.cc:1035] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:24.061592 15743 consensus_queue.cc:1035] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:24.062207 15742 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: TxnStatusTablet state changed. Reason: New leader 60b0d8d331334bf7ba22951fd8d2acaf. Latest consensus state: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.062222 15755 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: TxnStatusTablet state changed. Reason: New leader 60b0d8d331334bf7ba22951fd8d2acaf. Latest consensus state: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.062304 15742 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:24.062310 15755 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:24.063378 15743 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.063470 15743 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:24.063419 15788 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.063616 15742 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.063685 15742 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 1dfb245d1264418ab1083b5ac7e7b209: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:24.063560 15788 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 60b0d8d331334bf7ba22951fd8d2acaf: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:24.064003 15755 tablet_replica.cc:439] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 } } peers { permanent_uuid: "60b0d8d331334bf7ba22951fd8d2acaf" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43775 } } peers { permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 } } }
I20260619 20:44:24.064101 15755 tablet_replica.cc:442] T 9ca379c2945345abb68537c43ae48144 P 35ee02e01aea4708ac2113afa8ec1545: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:24.066655 15785 mvcc.cc:205] Tried to move back new op lower bound from 7298670035191873536 to 7298670035148423168. Current Snapshot: MvccSnapshot[applied={T|T < 7298670035191873536}]
I20260619 20:44:24.066946 15784 mvcc.cc:205] Tried to move back new op lower bound from 7298670035191873536 to 7298670035148423168. Current Snapshot: MvccSnapshot[applied={T|T < 7298670035191873536}]
I20260619 20:44:24.068810 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.065716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.13.195.66:43775)
0619 20:44:24.065756 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:24.065758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:24.065810 (+ 52us) server_negotiation.cc:407] Connection header received
0619 20:44:24.065870 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:24.065871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:24.065881 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:24.065946 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:24.066029 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.066844 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.067989 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.068296 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.068359 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.068533 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:24.068568 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:24.068623 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:24.068717 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:44:24.068739 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:44:24.074589 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.071704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:41377 (local address 127.13.195.66:43775)
0619 20:44:24.071757 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:24.071759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:24.071932 (+ 173us) server_negotiation.cc:407] Connection header received
0619 20:44:24.072004 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:24.072005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:24.072016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:24.072073 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:24.072285 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.072745 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.073887 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.074058 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.074109 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.074331 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:24.074354 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:24.074402 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:24.074505 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 20:44:24.074520 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:44:24.074848 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.071627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:43775 (local address 127.13.195.66:41377)
0619 20:44:24.071860 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:24.071871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:24.071933 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:24.072093 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:24.072094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:24.072096 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:24.072218 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:24.072220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.072823 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.072824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:24.073801 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:24.073803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.074131 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.074132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:24.074233 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.074249 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:24.074407 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:24.074408 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:24.074499 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 20:44:24.074544 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":93,"threads_started":1}
I20260619 20:44:24.077023 15665 raft_consensus.cc:1261] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Refusing update from remote peer 60b0d8d331334bf7ba22951fd8d2acaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:24.077231 15381 raft_consensus.cc:1261] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Refusing update from remote peer 60b0d8d331334bf7ba22951fd8d2acaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:24.077310 15743 consensus_queue.cc:1035] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "35ee02e01aea4708ac2113afa8ec1545" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:24.077615 15743 consensus_queue.cc:1035] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfb245d1264418ab1083b5ac7e7b209" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 44355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:24.091647 15218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:24.091763 15218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:24.092468 15218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a9018e255594a6e957722ed88cb9709: Sending DeleteTablet for 3 replicas of tablet 66c9d479cc254567b3ffd2b49eba5bf8
I20260619 20:44:24.092753 15361 tablet_service.cc:1514] Processing DeleteTablet for tablet 66c9d479cc254567b3ffd2b49eba5bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:24 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:57262
I20260619 20:44:24.092788 15644 tablet_service.cc:1514] Processing DeleteTablet for tablet 66c9d479cc254567b3ffd2b49eba5bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:24 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:59680
I20260619 20:44:24.092785 15501 tablet_service.cc:1514] Processing DeleteTablet for tablet 66c9d479cc254567b3ffd2b49eba5bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:24 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:47976
I20260619 20:44:24.093017 15802 tablet_replica.cc:330] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: stopping tablet replica
I20260619 20:44:24.093096 15802 raft_consensus.cc:2229] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:24.093185 15801 tablet_replica.cc:330] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: stopping tablet replica
I20260619 20:44:24.093194 15802 raft_consensus.cc:2258] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:24.093307 15801 raft_consensus.cc:2229] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:24.093469 15801 raft_consensus.cc:2258] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:24.093477 15802 ts_tablet_manager.cc:1905] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:24.093777 15801 ts_tablet_manager.cc:1905] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:24.095546 15802 ts_tablet_manager.cc:1918] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:24.095675 15802 log.cc:1196] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/66c9d479cc254567b3ffd2b49eba5bf8
I20260619 20:44:24.095891 15801 ts_tablet_manager.cc:1918] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:24.095983 15802 ts_tablet_manager.cc:1939] T 66c9d479cc254567b3ffd2b49eba5bf8 P 35ee02e01aea4708ac2113afa8ec1545: Deleting consensus metadata
I20260619 20:44:24.096107 15801 log.cc:1196] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/66c9d479cc254567b3ffd2b49eba5bf8
I20260619 20:44:24.096295 15206 catalog_manager.cc:4841] TS 35ee02e01aea4708ac2113afa8ec1545 (127.13.195.67:35377): tablet 66c9d479cc254567b3ffd2b49eba5bf8 (table test-table [id=a55aeb99e5834e19b2b96f80f1ca329a]) successfully deleted
I20260619 20:44:24.096553 15801 ts_tablet_manager.cc:1939] T 66c9d479cc254567b3ffd2b49eba5bf8 P 60b0d8d331334bf7ba22951fd8d2acaf: Deleting consensus metadata
I20260619 20:44:24.096927 15203 catalog_manager.cc:4841] TS 60b0d8d331334bf7ba22951fd8d2acaf (127.13.195.66:43775): tablet 66c9d479cc254567b3ffd2b49eba5bf8 (table test-table [id=a55aeb99e5834e19b2b96f80f1ca329a]) successfully deleted
I20260619 20:44:24.098094 15800 tablet_replica.cc:330] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: stopping tablet replica
I20260619 20:44:24.098189 15800 raft_consensus.cc:2229] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:24.098234 15800 raft_consensus.cc:2258] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:24.098511 15800 ts_tablet_manager.cc:1905] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 20:44:24 dist-test-slave-n2td krb5kdc[15174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901863, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:24.100188 15800 ts_tablet_manager.cc:1918] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:24.100252 15800 log.cc:1196] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/66c9d479cc254567b3ffd2b49eba5bf8
I20260619 20:44:24.100478 15800 ts_tablet_manager.cc:1939] T 66c9d479cc254567b3ffd2b49eba5bf8 P 1dfb245d1264418ab1083b5ac7e7b209: Deleting consensus metadata
I20260619 20:44:24.100941 15204 catalog_manager.cc:4841] TS 1dfb245d1264418ab1083b5ac7e7b209 (127.13.195.65:44355): tablet 66c9d479cc254567b3ffd2b49eba5bf8 (table test-table [id=a55aeb99e5834e19b2b96f80f1ca329a]) successfully deleted
I20260619 20:44:24.103487 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.093747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.13.195.65:44355)
0619 20:44:24.093787 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:24.093789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:24.094043 (+ 254us) server_negotiation.cc:407] Connection header received
0619 20:44:24.094147 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:24.094149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:24.094164 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:24.094194 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:24.095057 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.095824 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.097012 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.097322 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.097369 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.100550 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:24.100558 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:24.100559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:24.100566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:24.102183 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:24.102623 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:24.102627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:24.102628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:24.102692 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:24.102910 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:24.102911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:24.102911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:24.103061 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:24.103205 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:24.103278 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 20:44:24.103377 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:44:24.103746 15410 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57270
I20260619 20:44:24.110958 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:24.104711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.13.195.126:42235)
0619 20:44:24.104766 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:24.104769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:24.104790 (+ 21us) server_negotiation.cc:407] Connection header received
0619 20:44:24.104849 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:24.104850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:24.104860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:24.104906 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:24.105389 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.105890 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.106655 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:24.106937 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:24.107021 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:24.108231 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:24.108233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:24.108233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:24.108241 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:24.109704 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:24.110080 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:24.110082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:24.110082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:24.110138 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:24.110354 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:24.110355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:24.110355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:24.110541 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:24.110645 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:24.110766 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 20:44:24.110851 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 20:44:24.111227 15218 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48190
I20260619 20:44:24.111850 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 15292
I20260619 20:44:24.118736 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 15436
I20260619 20:44:24.124105 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 15577
I20260619 20:44:24.129765 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 15187
2026-06-19T20:44:24Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2653 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:24 dist-test-slave-n2td krb5kdc[15809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:24 dist-test-slave-n2td krb5kdc[15809](info): set up 2 sockets
Jun 19 20:44:24 dist-test-slave-n2td krb5kdc[15809](info): commencing operation
krb5kdc: starting...
W20260619 20:44:25.169106 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:25.285593 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:34939
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:34939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:25.365510 15822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:25.365788 15822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:25.365829 15822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:25.367059 15822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:25.367139 15822 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:44:25.367161 15822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:25.367182 15822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:25.367213 15822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:25.367235 15822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:25.368746 15822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:34939
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:34939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:25.369053 15822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:25.369295 15822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:44:25.374414 15822 server_base.cc:1034] running on GCE node
W20260619 20:44:25.374342 15828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.374349 15830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.374348 15827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:25.374902 15822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:25.375265 15822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:25.376473 15822 hybrid_clock.cc:648] HybridClock initialized: now 1781901865376445 us; error 49 us; skew 500 ppm
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:25.379164 15822 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:25.379657 15822 webserver.cc:460] Webserver started at http://127.13.195.126:41869/ using document root <none> and password file <none>
I20260619 20:44:25.379900 15822 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:25.379947 15822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:25.380043 15822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:25.381100 15822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "1fddb696c56e4f29ba44967f67ff76c9"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "0a78f4de9374b8f304cae84ddfd856cf"
server_key_iv: "345bd1810d80947008ea18df15419b10"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.381413 15822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "1fddb696c56e4f29ba44967f67ff76c9"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "0a78f4de9374b8f304cae84ddfd856cf"
server_key_iv: "345bd1810d80947008ea18df15419b10"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.382812 15822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:25.383613 15837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.383819 15822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:25.383904 15822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "1fddb696c56e4f29ba44967f67ff76c9"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "0a78f4de9374b8f304cae84ddfd856cf"
server_key_iv: "345bd1810d80947008ea18df15419b10"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.383961 15822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:25.408121 15822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:25.410583 15822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:25.410701 15822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:25.415501 15822 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:34939
I20260619 20:44:25.415560 15900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:34939 every 8 connection(s)
I20260619 20:44:25.415894 15822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:25.416488 15901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:25.418864 15901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Bootstrap starting.
I20260619 20:44:25.419457 15901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:25.419739 15901 log.cc:824] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:25.420421 15901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: No bootstrap required, opened a new log
I20260619 20:44:25.420449 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 15822
I20260619 20:44:25.420563 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:25.420739 14093 external_mini_cluster.cc:1428] Setting key 2052def4b95e92d92ee0c267f5f27ce5
I20260619 20:44:25.422832 15901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fddb696c56e4f29ba44967f67ff76c9" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34939 } }
I20260619 20:44:25.422993 15901 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:25.423058 15901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fddb696c56e4f29ba44967f67ff76c9, State: Initialized, Role: FOLLOWER
I20260619 20:44:25.423209 15901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fddb696c56e4f29ba44967f67ff76c9" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34939 } }
I20260619 20:44:25.423301 15901 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:25.423346 15901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:25.423394 15901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:25.423949 15901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fddb696c56e4f29ba44967f67ff76c9" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34939 } }
I20260619 20:44:25.424082 15901 leader_election.cc:302] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fddb696c56e4f29ba44967f67ff76c9; no voters:
I20260619 20:44:25.424253 15901 leader_election.cc:288] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:25.424310 15906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:25.424495 15906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [term 1 LEADER]: Becoming Leader. State: Replica: 1fddb696c56e4f29ba44967f67ff76c9, State: Running, Role: LEADER
I20260619 20:44:25.424525 15901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:25.424645 15906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fddb696c56e4f29ba44967f67ff76c9" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34939 } }
I20260619 20:44:25.425248 15907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fddb696c56e4f29ba44967f67ff76c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fddb696c56e4f29ba44967f67ff76c9" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34939 } } }
I20260619 20:44:25.425329 15907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:25.425724 15908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fddb696c56e4f29ba44967f67ff76c9. Latest consensus state: current_term: 1 leader_uuid: "1fddb696c56e4f29ba44967f67ff76c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fddb696c56e4f29ba44967f67ff76c9" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34939 } } }
I20260619 20:44:25.425874 15908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9 [sys.catalog]: This master's current role is: LEADER
W20260619 20:44:25.426401 15922 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:44:25.426491 15922 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:44:25.426589 15915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:25.426771 15915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:25.430979 15915 catalog_manager.cc:1347] Generated new cluster ID: 0d086db6fe994f91aeb69608bc9c0f1e
I20260619 20:44:25.431043 15915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:25.432909 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.421254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47326 (local address 127.13.195.126:34939)
0619 20:44:25.421435 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.421445 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.421483 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:44:25.422821 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.422829 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.422902 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.423059 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.423837 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.425162 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.426286 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.426897 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.426997 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.429453 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.429457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.429458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.429470 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.431298 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.431677 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.431680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.431680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.431782 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.431996 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.431997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.431998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.432239 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.432414 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.432511 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.432756 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260619 20:44:25.465333 15915 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:25.465935 15915 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:25.474093 15915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Generated new TSK 0
I20260619 20:44:25.474363 15915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:25.505569 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34939
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:25.612394 15928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:25.612607 15928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:25.612632 15928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:25.613935 15928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:25.613996 15928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:25.614056 15928 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:44:25.614090 15928 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:25.614121 15928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:25.615895 15928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:34939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:25.616175 15928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:25.616448 15928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:25.621793 15933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.621767 15936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.621798 15934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:25.621855 15928 server_base.cc:1034] running on GCE node
I20260619 20:44:25.622244 15928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:25.622501 15928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:25.623661 15928 hybrid_clock.cc:648] HybridClock initialized: now 1781901865623648 us; error 29 us; skew 500 ppm
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:25.626367 15928 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:25.626874 15928 webserver.cc:460] Webserver started at http://127.13.195.65:45563/ using document root <none> and password file <none>
I20260619 20:44:25.627125 15928 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:25.627175 15928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:25.627336 15928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:25.628523 15928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "498ceb5c83f54a548a6fe71714ccbac0"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "382b793a5637ecab8a268242e3b78072"
server_key_iv: "e05029008cbcf6f4c5b719831f186938"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.628927 15928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "498ceb5c83f54a548a6fe71714ccbac0"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "382b793a5637ecab8a268242e3b78072"
server_key_iv: "e05029008cbcf6f4c5b719831f186938"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.630366 15928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:25.631313 15943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.631491 15928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:25.631606 15928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "498ceb5c83f54a548a6fe71714ccbac0"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "382b793a5637ecab8a268242e3b78072"
server_key_iv: "e05029008cbcf6f4c5b719831f186938"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.631697 15928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:25.645651 15928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:25.647799 15928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:25.648010 15928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:25.648597 15928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:25.648669 15928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.648720 15928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:25.648752 15928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:25.656981 15928 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:39377
I20260619 20:44:25.657052 16060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:39377 every 8 connection(s)
I20260619 20:44:25.657434 15928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:25.659123 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.649171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:45015 (local address 127.13.195.126:34939)
0619 20:44:25.649212 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.649214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.649478 (+ 264us) server_negotiation.cc:407] Connection header received
0619 20:44:25.650036 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.650038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.650051 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.650109 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.650969 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.651770 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.652880 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.653099 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.653178 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.655967 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.655968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.655969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.655977 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.657662 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.658150 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.658151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.658152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.658204 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.658528 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.658529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.658529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.658642 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.658697 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.658970 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.659060 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:44:25.659291 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.649173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34939 (local address 127.13.195.65:45015)
0619 20:44:25.649343 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:25.649377 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:25.649918 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:25.650188 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:25.650191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:25.650588 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:25.650845 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.650851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.651865 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.651867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.652785 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.652787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.653184 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.653185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.653407 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.654046 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:25.654066 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:25.655860 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:25.657787 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.657793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.657796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.658083 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.658268 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.658268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.658268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.658406 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.658703 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:25.658708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:25.658829 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 20:44:25.658925 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:44:25.659074 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260619 20:44:25.659852 16061 heartbeater.cc:347] Connected to a master server at 127.13.195.126:34939
I20260619 20:44:25.659936 16061 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:25.660140 16061 heartbeater.cc:511] Master 127.13.195.126:34939 requested a full tablet report, sending...
I20260619 20:44:25.660632 15853 ts_manager.cc:194] Registered new tserver with Master: 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377)
I20260619 20:44:25.661505 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 15928
I20260619 20:44:25.661530 15853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:45015
I20260619 20:44:25.661607 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:25.661773 14093 external_mini_cluster.cc:1428] Setting key 120153107c1dc681a00ca868c99daa58
I20260619 20:44:25.667434 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.660893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34939 (local address 127.13.195.65:52219)
0619 20:44:25.661026 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:25.661036 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:25.661092 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:25.661230 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:25.661231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:25.661462 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:25.661553 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.661556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.662537 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.662539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.663296 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.663298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.663779 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.663781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.663879 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.664285 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:25.664291 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:25.664734 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:25.666495 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.666496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.666496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.666679 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.666887 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.666888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.666888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.666935 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.667201 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:25.667201 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:25.667237 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 20:44:25.667287 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:44:25.667364 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":51,"thread_start_us":132,"threads_started":1}
I20260619 20:44:25.667526 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.660961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:52219 (local address 127.13.195.126:34939)
0619 20:44:25.661010 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.661012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.661078 (+ 66us) server_negotiation.cc:407] Connection header received
0619 20:44:25.661153 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.661153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.661163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.661220 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.661646 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.662462 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.663385 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.663698 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.663771 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.664826 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.664828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.664828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.664835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.666390 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.666756 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.666757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.666758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.666819 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.666992 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.666993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.666994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.667133 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.667215 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.667316 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.667399 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:25.688727 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34939
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:25.769912 16070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:25.770188 16070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:25.770233 16070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:25.771909 16070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:25.771962 16070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:25.771988 16070 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:44:25.772001 16070 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:25.772019 16070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:25.773434 16070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:34939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:25.773644 16070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:25.773826 16070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:25.779019 16078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.779035 16076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.779014 16075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:25.779228 16070 server_base.cc:1034] running on GCE node
I20260619 20:44:25.779448 16070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:25.779721 16070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:25.780884 16070 hybrid_clock.cc:648] HybridClock initialized: now 1781901865780867 us; error 54 us; skew 500 ppm
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:25.783425 16070 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:25.783895 16070 webserver.cc:460] Webserver started at http://127.13.195.66:43625/ using document root <none> and password file <none>
I20260619 20:44:25.784086 16070 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:25.784128 16070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:25.784220 16070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:25.785287 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "e3bb624ecf6b4059b231729c48e2529d"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "c1a03efa439caec715c3684cbf17f4c6"
server_key_iv: "4d1f97c2e7a0d16746c41ba754c95406"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.785629 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "e3bb624ecf6b4059b231729c48e2529d"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "c1a03efa439caec715c3684cbf17f4c6"
server_key_iv: "4d1f97c2e7a0d16746c41ba754c95406"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.787119 16070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:25.787854 16085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.788021 16070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:25.788115 16070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "e3bb624ecf6b4059b231729c48e2529d"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "c1a03efa439caec715c3684cbf17f4c6"
server_key_iv: "4d1f97c2e7a0d16746c41ba754c95406"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.788172 16070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:25.795362 16070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:25.797153 16070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:25.797277 16070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:25.797716 16070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:25.797755 16070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.797777 16070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:25.797832 16070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.804733 16070 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:43763
I20260619 20:44:25.804807 16201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:43763 every 8 connection(s)
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:25.805157 16070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:25.807978 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.798213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:35349 (local address 127.13.195.126:34939)
0619 20:44:25.798285 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.798288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.798480 (+ 192us) server_negotiation.cc:407] Connection header received
0619 20:44:25.798986 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.798988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.799002 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.799082 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.799957 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.800806 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.802392 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.802778 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.802858 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.805590 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.805592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.805592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.805600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.806846 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.807208 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.807208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.807209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.807259 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.807468 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.807469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.807469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.807568 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.807617 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.807846 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.807914 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:44:25.808146 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.798171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34939 (local address 127.13.195.66:35349)
0619 20:44:25.798324 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:25.798364 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:25.798889 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:25.799178 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:25.799182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:25.799550 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:25.799838 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.799844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.800911 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.800912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.802300 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.802303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.802859 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.802860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.802966 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.803527 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:25.803539 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:25.805474 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:25.806920 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.806925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.806927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.807141 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.807301 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.807301 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.807301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.807385 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.807624 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:25.807628 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:25.807740 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 20:44:25.807824 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:44:25.807940 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260619 20:44:25.808678 16202 heartbeater.cc:347] Connected to a master server at 127.13.195.126:34939
I20260619 20:44:25.808758 16202 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:25.808929 16202 heartbeater.cc:511] Master 127.13.195.126:34939 requested a full tablet report, sending...
I20260619 20:44:25.809279 15854 ts_manager.cc:194] Registered new tserver with Master: e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763)
I20260619 20:44:25.809749 15854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:35349
I20260619 20:44:25.813719 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 16070
I20260619 20:44:25.813822 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:25.814061 14093 external_mini_cluster.cc:1428] Setting key eb8a14d069b684ed3fe94266953ddeec
I20260619 20:44:25.815326 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.809698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:60721 (local address 127.13.195.126:34939)
0619 20:44:25.809743 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.809745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.809877 (+ 132us) server_negotiation.cc:407] Connection header received
0619 20:44:25.809974 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.809975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.809985 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.810044 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.810388 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.810941 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.811606 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.811773 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.811821 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.812616 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.812617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.812617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.812623 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.814001 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.814453 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.814455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.814455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.814522 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.814740 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.814740 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.814741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.814902 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.814977 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.815141 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.815235 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:44:25.815364 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.809637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34939 (local address 127.13.195.66:60721)
0619 20:44:25.809816 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:25.809828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:25.809908 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:25.810064 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:25.810065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:25.810258 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:25.810317 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.810318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.811036 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.811037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.811535 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.811537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.811832 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.811832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.811896 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.812262 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:25.812266 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:25.812541 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:25.814080 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.814081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.814082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.814328 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.814599 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.814599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.814600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.814666 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.814979 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:25.814980 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:25.815028 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 20:44:25.815116 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:44:25.815234 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:25.839308 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34939
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:25.921744 16211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:25.921962 16211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:25.921988 16211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:25.923108 16211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:25.923146 16211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:25.923166 16211 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:44:25.923177 16211 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:25.923194 16211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:25.924863 16211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:34939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:25.925058 16211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:25.925244 16211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:25.930493 16219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.930514 16217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:25.930493 16216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:25.930709 16211 server_base.cc:1034] running on GCE node
I20260619 20:44:25.931013 16211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:25.931270 16211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:25.932439 16211 hybrid_clock.cc:648] HybridClock initialized: now 1781901865932424 us; error 32 us; skew 500 ppm
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:25.934995 16211 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:25.935478 16211 webserver.cc:460] Webserver started at http://127.13.195.67:43131/ using document root <none> and password file <none>
I20260619 20:44:25.935724 16211 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:25.935794 16211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:25.935943 16211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:25.937039 16211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "9ac309a295e142ddaaecd40b32453a93"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "fa897f715b71bc7e6835610d62d459b9"
server_key_iv: "ad818e96013e251fe5aef72372edb88d"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.937393 16211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "9ac309a295e142ddaaecd40b32453a93"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "fa897f715b71bc7e6835610d62d459b9"
server_key_iv: "ad818e96013e251fe5aef72372edb88d"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.938767 16211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:25.939577 16226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.939725 16211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:25.939829 16211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "9ac309a295e142ddaaecd40b32453a93"
format_stamp: "Formatted at 2026-06-19 20:44:25 on dist-test-slave-n2td"
server_key: "fa897f715b71bc7e6835610d62d459b9"
server_key_iv: "ad818e96013e251fe5aef72372edb88d"
server_key_version: "encryptionkey@0"
I20260619 20:44:25.939903 16211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:25.953624 16211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:25.956108 16211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:25.956285 16211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:25.956810 16211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:25.956873 16211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:25.956920 16211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:25.956945 16211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:25.964310 16211 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:40035
I20260619 20:44:25.964359 16342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:40035 every 8 connection(s)
I20260619 20:44:25.964771 16211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:25.966653 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.957378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:40847 (local address 127.13.195.126:34939)
0619 20:44:25.957453 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.957456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.957686 (+ 230us) server_negotiation.cc:407] Connection header received
0619 20:44:25.958201 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.958202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.958215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.958302 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.958989 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.959848 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.961016 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.961215 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.961305 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.963756 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.963758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.963758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.963766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.965458 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.965825 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.965826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.965827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.965879 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.966067 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.966067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.966067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.966172 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.966222 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.966513 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.966589 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:44:25.966836 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.957342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34939 (local address 127.13.195.67:40847)
0619 20:44:25.957556 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:25.957597 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:25.958103 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:25.958377 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:25.958381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:25.958711 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:25.958912 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.958917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.959946 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.959947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.960868 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.960871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.961337 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.961338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.961487 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.962061 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:25.962073 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:25.963649 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:25.965537 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.965541 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.965543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.965757 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.965920 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.965921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.965921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.966017 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.966231 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:25.966236 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:25.966346 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 20:44:25.966474 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 20:44:25.966629 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":146,"threads_started":1}
I20260619 20:44:25.967356 16343 heartbeater.cc:347] Connected to a master server at 127.13.195.126:34939
I20260619 20:44:25.967455 16343 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:25.967649 16343 heartbeater.cc:511] Master 127.13.195.126:34939 requested a full tablet report, sending...
I20260619 20:44:25.967988 15854 ts_manager.cc:194] Registered new tserver with Master: 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:25.968547 15854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:40847
I20260619 20:44:25.973655 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.968465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:48867 (local address 127.13.195.126:34939)
0619 20:44:25.968506 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.968508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.968573 (+ 65us) server_negotiation.cc:407] Connection header received
0619 20:44:25.968649 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.968651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.968661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.968709 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.969088 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.969555 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.970240 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.970505 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.970563 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.971430 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.971431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.971432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.971438 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.972528 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.972872 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.972873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.972873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.972927 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.973147 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.973148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.973148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.973258 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.973328 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.973486 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.973563 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:44:25.973668 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.968401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34939 (local address 127.13.195.67:48867)
0619 20:44:25.968514 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:25.968524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:25.968584 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:25.968750 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:25.968750 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:25.968945 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:25.969006 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.969007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.969648 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.969649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.970158 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:25.970159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.970561 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.970562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:25.970643 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.971063 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:25.971067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:25.971326 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:25.972609 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.972610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.972611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.972785 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.973021 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:25.973022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:25.973022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:25.973082 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:25.973333 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:25.973335 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:25.973378 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:44:25.973462 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:44:25.973565 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":73,"threads_started":1}
I20260619 20:44:25.974256 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 16211
I20260619 20:44:25.974359 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:25.974575 14093 external_mini_cluster.cc:1428] Setting key d0a3555b715b9654421f4b2748fe7393
I20260619 20:44:25.975629 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:44:25 dist-test-slave-n2td krb5kdc[15809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:25.997656 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:25.990947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.13.195.126:34939)
0619 20:44:25.991013 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:25.991015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:25.991034 (+ 19us) server_negotiation.cc:407] Connection header received
0619 20:44:25.991061 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:25.991062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:25.991070 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:25.991128 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:25.991671 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.992218 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.992943 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:25.993129 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:25.993179 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:25.995518 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:25.995519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:25.995520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:25.995526 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:25.996719 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.997051 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.997052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.997052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.997101 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:25.997260 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:25.997261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:25.997261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:25.997359 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:25.997430 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:25.997530 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 20:44:25.997589 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:44:25.999402 15854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:26.001019 15854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:26.007946 16363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.003439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39377 (local address 127.0.0.1:44264)
0619 20:44:26.003684 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.003696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.003939 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.004195 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.004195 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.004198 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:26.004240 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.004241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.005481 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.005482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.006643 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.006644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.007589 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.007589 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.007684 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.007702 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.007786 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.007855 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":109,"threads_started":1}
I20260619 20:44:26.007967 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.003703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.13.195.65:39377)
0619 20:44:26.003897 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.003903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.003916 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:44:26.004052 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.004055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.004071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.004136 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:26.004315 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.005391 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.006732 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.007504 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.007584 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.007654 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.007815 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.007858 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260619 20:44:26.009027 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.003360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40035 (local address 127.0.0.1:50044)
0619 20:44:26.003608 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.003656 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.003725 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.004076 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.004079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.004081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:26.004164 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.004166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.006512 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.006513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.007583 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.007585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.008774 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.008775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.008846 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.008849 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.008905 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.008947 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":76,"threads_started":1}
I20260619 20:44:26.009048 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.003484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.13.195.67:40035)
0619 20:44:26.003814 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.003820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.003832 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:44:26.003874 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.003877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.003893 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.003990 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:26.004248 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.005390 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.007667 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.008613 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.008683 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.008738 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.008928 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.008966 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":90,"threads_started":1}
I20260619 20:44:26.009181 15994 tablet_service.cc:1467] Processing CreateTablet for tablet e3307b3a3db44950a634b657e512981c (DEFAULT_TABLE table=test-table [id=f1e85aa245094ce098be3d129906814c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:26.009567 15994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3307b3a3db44950a634b657e512981c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.009791 16276 tablet_service.cc:1467] Processing CreateTablet for tablet e3307b3a3db44950a634b657e512981c (DEFAULT_TABLE table=test-table [id=f1e85aa245094ce098be3d129906814c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:26.010107 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3307b3a3db44950a634b657e512981c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.012727 16366 tablet_bootstrap.cc:492] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Bootstrap starting.
I20260619 20:44:26.012858 16367 tablet_bootstrap.cc:492] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Bootstrap starting.
I20260619 20:44:26.013188 16360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.003579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:43763 (local address 127.0.0.1:41982)
0619 20:44:26.003959 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.003970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.004026 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.004279 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.004280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.004282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:26.004330 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.004331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.005231 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.005232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.006638 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.006641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.012865 (+ 6224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.012866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.012939 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.012944 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.013040 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.013102 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260619 20:44:26.013259 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.003791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.13.195.66:43763)
0619 20:44:26.003967 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.003972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.004013 (+ 41us) server_negotiation.cc:407] Connection header received
0619 20:44:26.004117 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.004119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.004135 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.004207 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:26.004410 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.005133 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.007693 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.008433 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.008557 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.008755 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.013066 (+ 4311us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.013129 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260619 20:44:26.013460 16366 tablet_bootstrap.cc:654] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.013799 16366 log.cc:824] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:26.013890 16367 tablet_bootstrap.cc:654] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.014201 16367 log.cc:824] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:26.014181 16135 tablet_service.cc:1467] Processing CreateTablet for tablet e3307b3a3db44950a634b657e512981c (DEFAULT_TABLE table=test-table [id=f1e85aa245094ce098be3d129906814c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:26.014552 16135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3307b3a3db44950a634b657e512981c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.014598 16366 tablet_bootstrap.cc:492] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: No bootstrap required, opened a new log
I20260619 20:44:26.014658 16366 ts_tablet_manager.cc:1397] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:26.015147 16367 tablet_bootstrap.cc:492] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: No bootstrap required, opened a new log
I20260619 20:44:26.015275 16367 ts_tablet_manager.cc:1397] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:44:26.016180 16366 raft_consensus.cc:348] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.016347 16366 raft_consensus.cc:374] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.016409 16366 raft_consensus.cc:729] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.016533 16366 consensus_queue.cc:260] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.016835 16061 heartbeater.cc:503] Master 127.13.195.126:34939 was elected leader, sending a full tablet report...
I20260619 20:44:26.016825 16366 ts_tablet_manager.cc:1428] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:26.017343 16367 raft_consensus.cc:348] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.017552 16367 raft_consensus.cc:374] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.017627 16367 raft_consensus.cc:729] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac309a295e142ddaaecd40b32453a93, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.017804 16367 consensus_queue.cc:260] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.018105 16367 ts_tablet_manager.cc:1428] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:44:26.018255 16343 heartbeater.cc:503] Master 127.13.195.126:34939 was elected leader, sending a full tablet report...
I20260619 20:44:26.018368 16372 tablet_bootstrap.cc:492] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Bootstrap starting.
I20260619 20:44:26.019145 16372 tablet_bootstrap.cc:654] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.019511 16372 log.cc:824] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:26.020345 16372 tablet_bootstrap.cc:492] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: No bootstrap required, opened a new log
I20260619 20:44:26.020414 16372 ts_tablet_manager.cc:1397] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:26.021785 16372 raft_consensus.cc:348] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.021916 16372 raft_consensus.cc:374] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.021938 16372 raft_consensus.cc:729] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3bb624ecf6b4059b231729c48e2529d, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.022075 16372 consensus_queue.cc:260] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.022334 16372 ts_tablet_manager.cc:1428] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:26.022408 16202 heartbeater.cc:503] Master 127.13.195.126:34939 was elected leader, sending a full tablet report...
W20260619 20:44:26.055869 16203 tablet.cc:2369] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:26.098693 16370 raft_consensus.cc:482] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:26.098814 16370 raft_consensus.cc:504] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.099108 16370 leader_election.cc:288] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035), e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763)
I20260619 20:44:26.101984 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.099445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:32785 (local address 127.13.195.67:40035)
0619 20:44:26.099495 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.099497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.099526 (+ 29us) server_negotiation.cc:407] Connection header received
0619 20:44:26.099554 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.099555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.099565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.099671 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:26.099850 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.100328 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.101254 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.101622 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.101705 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.101714 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.101867 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.101903 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:44:26.101979 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.099325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40035 (local address 127.13.195.65:32785)
0619 20:44:26.099387 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.099401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.099474 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.099687 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.099689 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.099690 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:26.099764 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.099766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.100414 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.100415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.101159 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.101161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.101705 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.101705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.101792 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.101795 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.101863 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.101903 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:44:26.102216 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.099441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:43763 (local address 127.13.195.65:42195)
0619 20:44:26.099642 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.099653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.099729 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.099888 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.099889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.099890 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:26.099972 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.099973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.100638 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.100639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.101352 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.101353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.101909 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.101910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.101995 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.101998 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.102065 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.102113 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":128,"threads_started":1}
I20260619 20:44:26.102309 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.099543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:42195 (local address 127.13.195.66:43763)
0619 20:44:26.099604 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.099606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.099715 (+ 109us) server_negotiation.cc:407] Connection header received
0619 20:44:26.099791 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.099791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.099799 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.099878 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:26.100042 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.100546 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.101443 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.101802 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.101904 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.101911 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.102191 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.102236 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:44:26.102308 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3307b3a3db44950a634b657e512981c" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93" is_pre_election: true
I20260619 20:44:26.102488 16296 raft_consensus.cc:2454] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 0.
I20260619 20:44:26.102595 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3307b3a3db44950a634b657e512981c" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bb624ecf6b4059b231729c48e2529d" is_pre_election: true
I20260619 20:44:26.102730 15944 leader_election.cc:302] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, 9ac309a295e142ddaaecd40b32453a93; no voters:
I20260619 20:44:26.102769 16155 raft_consensus.cc:2454] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 0.
I20260619 20:44:26.102866 16370 raft_consensus.cc:2781] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:26.102921 16370 raft_consensus.cc:482] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:26.102945 16370 raft_consensus.cc:3037] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.103626 16370 raft_consensus.cc:504] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.103757 16370 leader_election.cc:288] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 election: Requested vote from peers 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035), e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763)
I20260619 20:44:26.103891 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3307b3a3db44950a634b657e512981c" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93"
I20260619 20:44:26.103953 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3307b3a3db44950a634b657e512981c" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bb624ecf6b4059b231729c48e2529d"
I20260619 20:44:26.104001 16296 raft_consensus.cc:3037] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.104015 16155 raft_consensus.cc:3037] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.104830 16155 raft_consensus.cc:2454] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 1.
I20260619 20:44:26.104830 16296 raft_consensus.cc:2454] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 1.
I20260619 20:44:26.105037 15944 leader_election.cc:302] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, 9ac309a295e142ddaaecd40b32453a93; no voters:
I20260619 20:44:26.105137 16370 raft_consensus.cc:2781] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:26.105285 16370 raft_consensus.cc:686] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 LEADER]: Becoming Leader. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Running, Role: LEADER
I20260619 20:44:26.105356 16370 consensus_queue.cc:237] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } }
I20260619 20:44:26.106271 15854 catalog_manager.cc:5495] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } health_report { overall_health: HEALTHY } } }
Jun 19 20:44:26 dist-test-slave-n2td krb5kdc[15809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901865, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:26.146298 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.139173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34939 (local address 127.0.0.1:47330)
0619 20:44:26.139443 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.139456 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.139581 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.139787 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.139788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.140120 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:26.140170 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.140173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.140796 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.140797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.141322 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.141323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.141638 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.141639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.141691 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.142286 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:26.142292 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:26.143811 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:26.145435 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:26.145441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:26.145443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:26.145625 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:26.145788 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:26.145789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:26.145789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:26.145841 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:26.146021 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:26.146026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:26.146065 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.146116 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.146202 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":110,"threads_started":1}
I20260619 20:44:26.146346 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.139266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.13.195.126:34939)
0619 20:44:26.139421 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.139424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.139547 (+ 123us) server_negotiation.cc:407] Connection header received
0619 20:44:26.139657 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.139658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.139668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.139723 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:26.140243 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.140721 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.141393 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.141572 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.141624 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.143894 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:26.143895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:26.143895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:26.143903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:26.145366 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:26.145683 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:26.145684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:26.145684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:26.145732 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:26.145877 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:26.145878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:26.145878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:26.145972 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:26.146016 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.146159 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.146248 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":87}
I20260619 20:44:26.148519 15854 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:47330:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:44:26.148747 15854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:26.150300 15994 tablet_service.cc:1467] Processing CreateTablet for tablet 64014214d1384f26acc4dd8696f29106 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0b68b31f4c341daac0b0c0cb1723438]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:26.150310 16135 tablet_service.cc:1467] Processing CreateTablet for tablet 64014214d1384f26acc4dd8696f29106 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0b68b31f4c341daac0b0c0cb1723438]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:26.150506 16135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64014214d1384f26acc4dd8696f29106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.150509 15994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64014214d1384f26acc4dd8696f29106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.151719 16366 tablet_bootstrap.cc:492] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: Bootstrap starting.
I20260619 20:44:26.151772 16372 tablet_bootstrap.cc:492] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: Bootstrap starting.
I20260619 20:44:26.151882 16276 tablet_service.cc:1467] Processing CreateTablet for tablet 64014214d1384f26acc4dd8696f29106 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0b68b31f4c341daac0b0c0cb1723438]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:26.152031 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64014214d1384f26acc4dd8696f29106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.152356 16372 tablet_bootstrap.cc:654] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.152423 16366 tablet_bootstrap.cc:654] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.152977 16372 tablet_bootstrap.cc:492] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: No bootstrap required, opened a new log
I20260619 20:44:26.153017 16372 ts_tablet_manager.cc:1397] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:26.153182 16372 raft_consensus.cc:348] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.153265 16372 raft_consensus.cc:374] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.153297 16372 raft_consensus.cc:729] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3bb624ecf6b4059b231729c48e2529d, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.153338 16372 consensus_queue.cc:260] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.153429 16372 ts_tablet_manager.cc:1428] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:26.153491 16374 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.153599 16374 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:26.153721 16366 tablet_bootstrap.cc:492] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: No bootstrap required, opened a new log
I20260619 20:44:26.153782 16366 ts_tablet_manager.cc:1397] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:26.153777 16367 tablet_bootstrap.cc:492] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: Bootstrap starting.
I20260619 20:44:26.153934 16366 raft_consensus.cc:348] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.154019 16366 raft_consensus.cc:374] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.154050 16366 raft_consensus.cc:729] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.154101 16366 consensus_queue.cc:260] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.154191 16366 ts_tablet_manager.cc:1428] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:26.154238 16370 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.154349 16370 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:26.154361 16367 tablet_bootstrap.cc:654] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.155128 16367 tablet_bootstrap.cc:492] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: No bootstrap required, opened a new log
I20260619 20:44:26.155195 16367 ts_tablet_manager.cc:1397] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:26.155347 16367 raft_consensus.cc:348] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.155417 16367 raft_consensus.cc:374] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.155452 16367 raft_consensus.cc:729] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac309a295e142ddaaecd40b32453a93, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.155501 16367 consensus_queue.cc:260] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.155601 16367 ts_tablet_manager.cc:1428] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:26.155678 16371 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.155779 16371 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 20:44:26.158128 16062 tablet.cc:2369] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:44:26.215420 16344 tablet.cc:2369] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:26.348454 16370 raft_consensus.cc:482] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:26.348559 16370 raft_consensus.cc:504] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.348701 16370 leader_election.cc:288] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:26.348941 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64014214d1384f26acc4dd8696f29106" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bb624ecf6b4059b231729c48e2529d" is_pre_election: true
I20260619 20:44:26.349032 16155 raft_consensus.cc:2454] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 0.
I20260619 20:44:26.348968 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64014214d1384f26acc4dd8696f29106" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93" is_pre_election: true
I20260619 20:44:26.349116 16296 raft_consensus.cc:2454] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 0.
I20260619 20:44:26.349228 15944 leader_election.cc:302] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, e3bb624ecf6b4059b231729c48e2529d; no voters:
I20260619 20:44:26.349346 16370 raft_consensus.cc:2781] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:26.349386 16370 raft_consensus.cc:482] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:26.349404 16370 raft_consensus.cc:3037] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.350067 16370 raft_consensus.cc:504] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.350200 16370 leader_election.cc:288] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 election: Requested vote from peers e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:26.350384 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64014214d1384f26acc4dd8696f29106" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bb624ecf6b4059b231729c48e2529d"
I20260619 20:44:26.350391 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64014214d1384f26acc4dd8696f29106" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93"
I20260619 20:44:26.350490 16296 raft_consensus.cc:3037] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.350490 16155 raft_consensus.cc:3037] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.351227 16155 raft_consensus.cc:2454] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 1.
I20260619 20:44:26.351227 16296 raft_consensus.cc:2454] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 1.
I20260619 20:44:26.351387 15944 leader_election.cc:302] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, 9ac309a295e142ddaaecd40b32453a93; no voters:
I20260619 20:44:26.351529 16370 raft_consensus.cc:2781] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:26.351596 16370 raft_consensus.cc:686] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 LEADER]: Becoming Leader. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Running, Role: LEADER
I20260619 20:44:26.351653 16370 consensus_queue.cc:237] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.351958 16376 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: TxnStatusTablet state changed. Reason: New leader 498ceb5c83f54a548a6fe71714ccbac0. Latest consensus state: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.352082 16376 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:26.352255 16388 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:44:26.352380 16388 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:44:26.352240 15854 catalog_manager.cc:5495] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:26.382977 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.380450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.13.195.65:39377)
0619 20:44:26.380512 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.380515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.380524 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:26.380601 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.380602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.380623 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.380699 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:26.380841 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.381368 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.382119 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.382316 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.382400 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.382617 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.382715 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.382816 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.382868 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.382889 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:44:26.382978 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.380363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39377 (local address 127.0.0.1:44280)
0619 20:44:26.380434 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.380446 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.380525 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.380704 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.380705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.380707 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:26.380763 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.380765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.381457 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.381458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.382029 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.382031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.382402 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.382403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.382502 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.382517 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.382796 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.382807 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.382863 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.382912 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:44:26.384963 16296 raft_consensus.cc:1261] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Refusing update from remote peer 498ceb5c83f54a548a6fe71714ccbac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:26.385018 16155 raft_consensus.cc:1261] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Refusing update from remote peer 498ceb5c83f54a548a6fe71714ccbac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:26.385324 16370 consensus_queue.cc:1035] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:26.385465 16376 consensus_queue.cc:1035] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:26.386730 16371 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: TxnStatusTablet state changed. Reason: New leader 498ceb5c83f54a548a6fe71714ccbac0. Latest consensus state: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.386854 16371 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:26.386816 16374 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: TxnStatusTablet state changed. Reason: New leader 498ceb5c83f54a548a6fe71714ccbac0. Latest consensus state: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.386898 16374 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:26.387917 16376 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.388003 16376 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:26.388341 16370 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.388424 16370 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P 498ceb5c83f54a548a6fe71714ccbac0: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:26.388700 16371 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.388783 16371 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P 9ac309a295e142ddaaecd40b32453a93: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:26.389554 16374 tablet_replica.cc:439] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } } }
I20260619 20:44:26.389639 16374 tablet_replica.cc:442] T 64014214d1384f26acc4dd8696f29106 P e3bb624ecf6b4059b231729c48e2529d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:26.394560 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.392195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.13.195.65:39377)
0619 20:44:26.392230 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.392232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.392243 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:44:26.392320 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.392321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.392329 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.392393 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:26.392552 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.393019 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.393808 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.394047 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.394125 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.394287 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.394320 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.394387 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.394464 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.394483 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:44:26.399905 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.397424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:40693 (local address 127.13.195.65:39377)
0619 20:44:26.397468 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.397470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.397579 (+ 109us) server_negotiation.cc:407] Connection header received
0619 20:44:26.397637 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.397638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.397647 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.397695 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:26.397884 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.398315 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.399195 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.399374 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.399458 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.399649 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.399684 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.399746 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.399805 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.399828 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":19}
I20260619 20:44:26.399930 16402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.397336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39377 (local address 127.13.195.65:40693)
0619 20:44:26.397514 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.397522 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.397591 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.397695 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.397696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.397698 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:26.397809 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.397810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.398394 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.398395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.399049 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.399051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.399457 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.399458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.399529 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.399547 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.399746 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.399748 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.399801 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.399850 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260619 20:44:26.402047 16155 raft_consensus.cc:1261] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Refusing update from remote peer 498ceb5c83f54a548a6fe71714ccbac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:26.402070 16296 raft_consensus.cc:1261] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Refusing update from remote peer 498ceb5c83f54a548a6fe71714ccbac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:26.402247 16403 consensus_queue.cc:1035] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:26.402390 16370 consensus_queue.cc:1035] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:26.407588 16393 mvcc.cc:205] Tried to move back new op lower bound from 7298670044781449216 to 7298670043568119808. Current Snapshot: MvccSnapshot[applied={T|T < 7298670044781449216}]
I20260619 20:44:26.407635 16392 mvcc.cc:205] Tried to move back new op lower bound from 7298670044781449216 to 7298670043568119808. Current Snapshot: MvccSnapshot[applied={T|T < 7298670044781449216}]
I20260619 20:44:26.418220 15853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:26.418334 15853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:26.419054 15853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Sending DeleteTablet for 3 replicas of tablet e3307b3a3db44950a634b657e512981c
I20260619 20:44:26.419330 15994 tablet_service.cc:1514] Processing DeleteTablet for tablet e3307b3a3db44950a634b657e512981c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:26 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:44264
I20260619 20:44:26.419332 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet e3307b3a3db44950a634b657e512981c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:26 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:50044
I20260619 20:44:26.419545 16135 tablet_service.cc:1514] Processing DeleteTablet for tablet e3307b3a3db44950a634b657e512981c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:26 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:41982
I20260619 20:44:26.419569 15853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 20:44:26.419677 16408 tablet_replica.cc:330] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: stopping tablet replica
W20260619 20:44:26.419713 15853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:26.419785 16408 raft_consensus.cc:2229] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:26.419886 16408 raft_consensus.cc:2258] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:26.419942 16407 tablet_replica.cc:330] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: stopping tablet replica
I20260619 20:44:26.420181 16407 raft_consensus.cc:2229] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:26.420312 16407 raft_consensus.cc:2258] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:26.420555 16408 ts_tablet_manager.cc:1905] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:26.420768 16407 ts_tablet_manager.cc:1905] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:26.421240 15994 tablet_service.cc:1467] Processing CreateTablet for tablet dcd7869b206141c9a10ff3af7ff0ee24 (DEFAULT_TABLE table=test-table [id=901dddcd7beb4c4696f4c9fe5a3c9f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:26.421280 16135 tablet_service.cc:1467] Processing CreateTablet for tablet dcd7869b206141c9a10ff3af7ff0ee24 (DEFAULT_TABLE table=test-table [id=901dddcd7beb4c4696f4c9fe5a3c9f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:26.421350 15994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd7869b206141c9a10ff3af7ff0ee24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.421371 16135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd7869b206141c9a10ff3af7ff0ee24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.421742 16276 tablet_service.cc:1467] Processing CreateTablet for tablet dcd7869b206141c9a10ff3af7ff0ee24 (DEFAULT_TABLE table=test-table [id=901dddcd7beb4c4696f4c9fe5a3c9f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:26.421854 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcd7869b206141c9a10ff3af7ff0ee24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:26.422366 16408 ts_tablet_manager.cc:1918] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:26.422454 16407 ts_tablet_manager.cc:1918] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:26.422453 16408 log.cc:1196] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/e3307b3a3db44950a634b657e512981c
I20260619 20:44:26.422508 16407 log.cc:1196] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/e3307b3a3db44950a634b657e512981c
I20260619 20:44:26.422722 16408 ts_tablet_manager.cc:1939] T e3307b3a3db44950a634b657e512981c P 498ceb5c83f54a548a6fe71714ccbac0: Deleting consensus metadata
I20260619 20:44:26.422736 16407 ts_tablet_manager.cc:1939] T e3307b3a3db44950a634b657e512981c P 9ac309a295e142ddaaecd40b32453a93: Deleting consensus metadata
I20260619 20:44:26.423059 16409 tablet_replica.cc:330] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: stopping tablet replica
I20260619 20:44:26.423096 15841 catalog_manager.cc:4841] TS 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377): tablet e3307b3a3db44950a634b657e512981c (table test-table [id=f1e85aa245094ce098be3d129906814c]) successfully deleted
I20260619 20:44:26.423180 16366 tablet_bootstrap.cc:492] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: Bootstrap starting.
I20260619 20:44:26.423219 16409 raft_consensus.cc:2229] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:26.423271 16409 raft_consensus.cc:2258] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:26.423542 16366 tablet_bootstrap.cc:654] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.423689 16409 ts_tablet_manager.cc:1905] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:26.423846 16367 tablet_bootstrap.cc:492] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: Bootstrap starting.
I20260619 20:44:26.424234 16367 tablet_bootstrap.cc:654] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.424899 16367 tablet_bootstrap.cc:492] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: No bootstrap required, opened a new log
I20260619 20:44:26.424947 16367 ts_tablet_manager.cc:1397] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:26.425122 16367 raft_consensus.cc:348] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.425184 16367 raft_consensus.cc:374] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.425199 16367 raft_consensus.cc:729] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac309a295e142ddaaecd40b32453a93, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.425248 16367 consensus_queue.cc:260] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.425352 16367 ts_tablet_manager.cc:1428] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:26.425352 16409 ts_tablet_manager.cc:1918] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:26.425420 16409 log.cc:1196] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/e3307b3a3db44950a634b657e512981c
I20260619 20:44:26.425659 16409 ts_tablet_manager.cc:1939] T e3307b3a3db44950a634b657e512981c P e3bb624ecf6b4059b231729c48e2529d: Deleting consensus metadata
I20260619 20:44:26.426085 16366 tablet_bootstrap.cc:492] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: No bootstrap required, opened a new log
I20260619 20:44:26.426131 16366 ts_tablet_manager.cc:1397] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:44:26.426261 16366 raft_consensus.cc:348] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.426326 16366 raft_consensus.cc:374] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.426347 16366 raft_consensus.cc:729] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.426339 15838 catalog_manager.cc:4841] TS e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763): tablet e3307b3a3db44950a634b657e512981c (table test-table [id=f1e85aa245094ce098be3d129906814c]) successfully deleted
I20260619 20:44:26.426393 16366 consensus_queue.cc:260] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.426503 16366 ts_tablet_manager.cc:1428] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:26.425796 16372 tablet_bootstrap.cc:492] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: Bootstrap starting.
I20260619 20:44:26.426959 16372 tablet_bootstrap.cc:654] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:26.427342 15838 catalog_manager.cc:4841] TS 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035): tablet e3307b3a3db44950a634b657e512981c (table test-table [id=f1e85aa245094ce098be3d129906814c]) successfully deleted
I20260619 20:44:26.427627 16372 tablet_bootstrap.cc:492] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: No bootstrap required, opened a new log
I20260619 20:44:26.427683 16372 ts_tablet_manager.cc:1397] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:26.427820 16372 raft_consensus.cc:348] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.427898 16372 raft_consensus.cc:374] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:26.427920 16372 raft_consensus.cc:729] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3bb624ecf6b4059b231729c48e2529d, State: Initialized, Role: FOLLOWER
I20260619 20:44:26.427963 16372 consensus_queue.cc:260] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.428058 16372 ts_tablet_manager.cc:1428] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:26.553694 16374 raft_consensus.cc:482] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:26.553799 16374 raft_consensus.cc:504] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.554188 16374 leader_election.cc:288] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:26.557808 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.554392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40035 (local address 127.13.195.66:35775)
0619 20:44:26.554657 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.554668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.554761 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.554913 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.554914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.554916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:26.554995 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.554997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.555873 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.555874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.556916 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.556918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.557491 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.557492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.557561 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.557564 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.557666 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.557710 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":117,"threads_started":1}
I20260619 20:44:26.557834 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.554604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:35775 (local address 127.13.195.67:40035)
0619 20:44:26.554650 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.554652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.554745 (+ 93us) server_negotiation.cc:407] Connection header received
0619 20:44:26.554820 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.554821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.554831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.554879 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:26.555085 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.555784 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.557049 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.557400 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.557493 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.557502 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.557716 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.557763 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:44:26.557900 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.554534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:33009 (local address 127.13.195.65:39377)
0619 20:44:26.554592 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.554594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.554647 (+ 53us) server_negotiation.cc:407] Connection header received
0619 20:44:26.554783 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.554784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.554794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.554846 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:26.555009 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.555528 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.556985 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.557530 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.557612 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.557622 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.557774 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.557817 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:44:26.558159 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.554351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39377 (local address 127.13.195.66:33009)
0619 20:44:26.554567 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.554579 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.554677 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.554868 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.554869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.554871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:26.554931 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.554933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.555639 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.555640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.556870 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.556873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.557615 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.557615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.557689 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.557692 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.558018 (+ 326us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.558064 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260619 20:44:26.558265 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd7869b206141c9a10ff3af7ff0ee24" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93" is_pre_election: true
I20260619 20:44:26.558367 16296 raft_consensus.cc:2454] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 0.
I20260619 20:44:26.558360 16014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd7869b206141c9a10ff3af7ff0ee24" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498ceb5c83f54a548a6fe71714ccbac0" is_pre_election: true
I20260619 20:44:26.558506 16014 raft_consensus.cc:2454] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 0.
I20260619 20:44:26.558591 16086 leader_election.cc:302] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ac309a295e142ddaaecd40b32453a93, e3bb624ecf6b4059b231729c48e2529d; no voters:
I20260619 20:44:26.558737 16374 raft_consensus.cc:2781] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:26.558784 16374 raft_consensus.cc:482] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:26.558800 16374 raft_consensus.cc:3037] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.559468 16374 raft_consensus.cc:504] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.559625 16374 leader_election.cc:288] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 election: Requested vote from peers 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:26.559770 16014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd7869b206141c9a10ff3af7ff0ee24" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498ceb5c83f54a548a6fe71714ccbac0"
I20260619 20:44:26.559855 16014 raft_consensus.cc:3037] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.559850 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcd7869b206141c9a10ff3af7ff0ee24" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93"
I20260619 20:44:26.559932 16296 raft_consensus.cc:3037] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:26.560513 16014 raft_consensus.cc:2454] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 1.
I20260619 20:44:26.560688 16089 leader_election.cc:302] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, e3bb624ecf6b4059b231729c48e2529d; no voters:
I20260619 20:44:26.560745 16296 raft_consensus.cc:2454] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 1.
I20260619 20:44:26.560789 16374 raft_consensus.cc:2781] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:26.560923 16374 raft_consensus.cc:686] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 1 LEADER]: Becoming Leader. State: Replica: e3bb624ecf6b4059b231729c48e2529d, State: Running, Role: LEADER
I20260619 20:44:26.560990 16374 consensus_queue.cc:237] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:26.561767 15854 catalog_manager.cc:5495] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d reported cstate change: term changed from 0 to 1, leader changed from <none> to e3bb624ecf6b4059b231729c48e2529d (127.13.195.66). New cstate: current_term: 1 leader_uuid: "e3bb624ecf6b4059b231729c48e2529d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:26.572685 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.570168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.13.195.66:43763)
0619 20:44:26.570212 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.570215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.570224 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:26.570296 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.570296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.570315 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.570407 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:26.570538 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.571017 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.571889 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.572091 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.572175 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.572318 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.572438 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.572519 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.572572 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.572593 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:44:26.577828 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.575463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:55595 (local address 127.13.195.65:39377)
0619 20:44:26.575508 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.575510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.575595 (+ 85us) server_negotiation.cc:407] Connection header received
0619 20:44:26.575660 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.575661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.575672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.575722 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:26.575866 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.576318 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.577084 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.577293 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.577370 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.577566 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.577599 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.577666 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.577728 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.577745 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:44:26.577865 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.575373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39377 (local address 127.13.195.66:55595)
0619 20:44:26.575516 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.575528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.575611 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.575723 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.575724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.575726 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:26.575799 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.575800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.576421 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.576422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.576993 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.576995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.577388 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.577389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.577454 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.577467 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.577668 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.577670 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.577722 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.577761 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260619 20:44:26.580186 16296 raft_consensus.cc:1261] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Refusing update from remote peer e3bb624ecf6b4059b231729c48e2529d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:26.580225 16014 raft_consensus.cc:1261] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Refusing update from remote peer e3bb624ecf6b4059b231729c48e2529d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:26.580442 16415 consensus_queue.cc:1035] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:26.580580 16374 consensus_queue.cc:1035] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [LEADER]: Connected to new peer: Peer: permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:26.582486 16391 mvcc.cc:205] Tried to move back new op lower bound from 7298670045510750208 to 7298670045434482688. Current Snapshot: MvccSnapshot[applied={T|T < 7298670045510750208}]
I20260619 20:44:26.589030 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.586456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:58575 (local address 127.13.195.66:43763)
0619 20:44:26.586492 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:26.586494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:26.586502 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:26.586565 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:26.586566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:26.586578 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:26.586627 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:26.586741 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.587292 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.588360 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.588643 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.588705 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.588841 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.588866 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.588908 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:26.588956 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 20:44:26.588971 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:44:26.589067 16402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:26.586384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:43763 (local address 127.13.195.65:58575)
0619 20:44:26.586446 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:26.586458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:26.586502 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:26.586628 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:26.586629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:26.586631 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:26.586689 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.586690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.587412 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.587413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.588279 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:26.588281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:26.588713 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:26.588714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:26.588771 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:26.588778 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:26.588907 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:26.588908 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:26.588953 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:44:26.588994 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 20:44:26.589414 16392 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 20:44:28.114961 15853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:28.115067 15853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:28.115862 15853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Sending DeleteTablet for 3 replicas of tablet dcd7869b206141c9a10ff3af7ff0ee24
I20260619 20:44:28.116109 15993 tablet_service.cc:1514] Processing DeleteTablet for tablet dcd7869b206141c9a10ff3af7ff0ee24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:28 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:44264
I20260619 20:44:28.116147 16135 tablet_service.cc:1514] Processing DeleteTablet for tablet dcd7869b206141c9a10ff3af7ff0ee24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:28 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:41982
I20260619 20:44:28.116343 16447 tablet_replica.cc:330] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: stopping tablet replica
I20260619 20:44:28.116604 16447 raft_consensus.cc:2229] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:28.116657 16447 raft_consensus.cc:2258] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:28.116888 16447 ts_tablet_manager.cc:1905] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:28.117604 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet dcd7869b206141c9a10ff3af7ff0ee24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:28 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:50044
I20260619 20:44:28.118738 16448 tablet_replica.cc:330] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: stopping tablet replica
I20260619 20:44:28.118855 16448 raft_consensus.cc:2229] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:28.118875 16450 tablet_replica.cc:330] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: stopping tablet replica
I20260619 20:44:28.118927 16450 raft_consensus.cc:2229] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:28.119001 16450 raft_consensus.cc:2258] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:28.119027 16448 raft_consensus.cc:2258] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:28.119228 16448 ts_tablet_manager.cc:1905] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:28.119278 16450 ts_tablet_manager.cc:1905] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:28.120841 16450 ts_tablet_manager.cc:1918] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:44:28.120841 16448 ts_tablet_manager.cc:1918] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:44:28.120903 16450 log.cc:1196] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/dcd7869b206141c9a10ff3af7ff0ee24
I20260619 20:44:28.120903 16448 log.cc:1196] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/dcd7869b206141c9a10ff3af7ff0ee24
I20260619 20:44:28.121040 16447 ts_tablet_manager.cc:1918] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:44:28.121110 16447 log.cc:1196] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/dcd7869b206141c9a10ff3af7ff0ee24
I20260619 20:44:28.121148 16448 ts_tablet_manager.cc:1939] T dcd7869b206141c9a10ff3af7ff0ee24 P e3bb624ecf6b4059b231729c48e2529d: Deleting consensus metadata
I20260619 20:44:28.121148 16450 ts_tablet_manager.cc:1939] T dcd7869b206141c9a10ff3af7ff0ee24 P 9ac309a295e142ddaaecd40b32453a93: Deleting consensus metadata
I20260619 20:44:28.121447 16447 ts_tablet_manager.cc:1939] T dcd7869b206141c9a10ff3af7ff0ee24 P 498ceb5c83f54a548a6fe71714ccbac0: Deleting consensus metadata
I20260619 20:44:28.121517 15838 catalog_manager.cc:4841] TS e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763): tablet dcd7869b206141c9a10ff3af7ff0ee24 (table test-table [id=901dddcd7beb4c4696f4c9fe5a3c9f01]) successfully deleted
I20260619 20:44:28.121593 15838 catalog_manager.cc:4841] TS 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035): tablet dcd7869b206141c9a10ff3af7ff0ee24 (table test-table [id=901dddcd7beb4c4696f4c9fe5a3c9f01]) successfully deleted
I20260619 20:44:28.121814 15841 catalog_manager.cc:4841] TS 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377): tablet dcd7869b206141c9a10ff3af7ff0ee24 (table test-table [id=901dddcd7beb4c4696f4c9fe5a3c9f01]) successfully deleted
Jun 19 20:44:28 dist-test-slave-n2td krb5kdc[15809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:44:28 dist-test-slave-n2td krb5kdc[15809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:28.145340 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:28.133376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.13.195.126:34939)
0619 20:44:28.133511 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:28.133513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:28.133632 (+ 119us) server_negotiation.cc:407] Connection header received
0619 20:44:28.133685 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:28.133686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:28.133698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:28.133760 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:28.134304 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:28.134930 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:28.135637 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:28.135873 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:28.135926 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:28.142816 (+ 6890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:28.142817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:28.142817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:28.142824 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:28.144356 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:28.144668 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:28.144668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:28.144669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:28.144721 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:28.144875 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:28.144875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:28.144875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:28.144978 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:28.145054 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:28.145153 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 20:44:28.145216 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260619 20:44:28.146736 15853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:28.146884 15853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:28.148171 15993 tablet_service.cc:1467] Processing CreateTablet for tablet adbc98875ae147f4910e50ecc5dec292 (DEFAULT_TABLE table=test-table [id=a9fd3a11f8db4265b3b2eca65273ce1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:28.148217 16276 tablet_service.cc:1467] Processing CreateTablet for tablet adbc98875ae147f4910e50ecc5dec292 (DEFAULT_TABLE table=test-table [id=a9fd3a11f8db4265b3b2eca65273ce1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:28.148237 16135 tablet_service.cc:1467] Processing CreateTablet for tablet adbc98875ae147f4910e50ecc5dec292 (DEFAULT_TABLE table=test-table [id=a9fd3a11f8db4265b3b2eca65273ce1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:28.148314 15993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adbc98875ae147f4910e50ecc5dec292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:28.148350 16135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adbc98875ae147f4910e50ecc5dec292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:28.148348 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adbc98875ae147f4910e50ecc5dec292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:28.149461 16458 tablet_bootstrap.cc:492] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: Bootstrap starting.
I20260619 20:44:28.149714 16459 tablet_bootstrap.cc:492] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: Bootstrap starting.
I20260619 20:44:28.149904 16458 tablet_bootstrap.cc:654] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:28.150005 16460 tablet_bootstrap.cc:492] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: Bootstrap starting.
I20260619 20:44:28.150087 16459 tablet_bootstrap.cc:654] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:28.150465 16460 tablet_bootstrap.cc:654] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:28.150610 16458 tablet_bootstrap.cc:492] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: No bootstrap required, opened a new log
I20260619 20:44:28.150676 16458 ts_tablet_manager.cc:1397] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:28.150789 16458 raft_consensus.cc:348] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.150879 16458 raft_consensus.cc:374] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:28.150907 16458 raft_consensus.cc:729] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Initialized, Role: FOLLOWER
I20260619 20:44:28.150959 16458 consensus_queue.cc:260] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.151073 16458 ts_tablet_manager.cc:1428] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:28.151109 16460 tablet_bootstrap.cc:492] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: No bootstrap required, opened a new log
I20260619 20:44:28.151162 16460 ts_tablet_manager.cc:1397] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:28.151302 16460 raft_consensus.cc:348] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.151377 16460 raft_consensus.cc:374] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:28.151397 16460 raft_consensus.cc:729] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac309a295e142ddaaecd40b32453a93, State: Initialized, Role: FOLLOWER
I20260619 20:44:28.151454 16460 consensus_queue.cc:260] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.151602 16459 tablet_bootstrap.cc:492] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: No bootstrap required, opened a new log
I20260619 20:44:28.151659 16459 ts_tablet_manager.cc:1397] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:28.151638 16460 ts_tablet_manager.cc:1428] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:28.151785 16459 raft_consensus.cc:348] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.151883 16459 raft_consensus.cc:374] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:28.151916 16459 raft_consensus.cc:729] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3bb624ecf6b4059b231729c48e2529d, State: Initialized, Role: FOLLOWER
I20260619 20:44:28.151968 16459 consensus_queue.cc:260] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.152067 16459 ts_tablet_manager.cc:1428] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:28.164368 16428 raft_consensus.cc:482] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:28.164453 16428 raft_consensus.cc:504] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.164589 16428 leader_election.cc:288] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:28.164772 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbc98875ae147f4910e50ecc5dec292" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bb624ecf6b4059b231729c48e2529d" is_pre_election: true
I20260619 20:44:28.164799 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbc98875ae147f4910e50ecc5dec292" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93" is_pre_election: true
I20260619 20:44:28.164875 16155 raft_consensus.cc:2454] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 0.
I20260619 20:44:28.164911 16296 raft_consensus.cc:2454] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 0.
I20260619 20:44:28.165031 15944 leader_election.cc:302] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, e3bb624ecf6b4059b231729c48e2529d; no voters:
I20260619 20:44:28.165122 16428 raft_consensus.cc:2781] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:28.165163 16428 raft_consensus.cc:482] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:28.165194 16428 raft_consensus.cc:3037] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:28.165896 16428 raft_consensus.cc:504] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.166033 16428 leader_election.cc:288] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 election: Requested vote from peers e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:28.166177 16155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbc98875ae147f4910e50ecc5dec292" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3bb624ecf6b4059b231729c48e2529d"
I20260619 20:44:28.166249 16155 raft_consensus.cc:3037] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:28.166196 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbc98875ae147f4910e50ecc5dec292" candidate_uuid: "498ceb5c83f54a548a6fe71714ccbac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93"
I20260619 20:44:28.166293 16296 raft_consensus.cc:3037] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:28.166977 16155 raft_consensus.cc:2454] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 1.
I20260619 20:44:28.166987 16296 raft_consensus.cc:2454] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ceb5c83f54a548a6fe71714ccbac0 in term 1.
I20260619 20:44:28.167151 15944 leader_election.cc:302] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, 9ac309a295e142ddaaecd40b32453a93; no voters:
I20260619 20:44:28.167233 16428 raft_consensus.cc:2781] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:28.167284 16428 raft_consensus.cc:686] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 LEADER]: Becoming Leader. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Running, Role: LEADER
I20260619 20:44:28.167338 16428 consensus_queue.cc:237] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.167939 15854 catalog_manager.cc:5495] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "498ceb5c83f54a548a6fe71714ccbac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:28.175856 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:28.173563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44290 (local address 127.13.195.65:39377)
0619 20:44:28.173715 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:28.173717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:28.173722 (+ 5us) server_negotiation.cc:407] Connection header received
0619 20:44:28.173739 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:28.173740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:28.173749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:28.173814 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:28.173974 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:28.174501 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:28.175270 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:28.175461 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:28.175512 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:28.175622 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:28.175648 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:28.175686 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:28.175743 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 20:44:28.175759 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":111,"threads_started":1}
I20260619 20:44:28.177893 16296 raft_consensus.cc:1261] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Refusing update from remote peer 498ceb5c83f54a548a6fe71714ccbac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:28.178038 16155 raft_consensus.cc:1261] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Refusing update from remote peer 498ceb5c83f54a548a6fe71714ccbac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:28.178102 16428 consensus_queue.cc:1035] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:28.178318 16428 consensus_queue.cc:1035] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:28.187700 15854 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:28.187803 15854 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:28.188417 15854 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Sending DeleteTablet for 3 replicas of tablet adbc98875ae147f4910e50ecc5dec292
I20260619 20:44:28.188635 15993 tablet_service.cc:1514] Processing DeleteTablet for tablet adbc98875ae147f4910e50ecc5dec292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:28 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:44264
I20260619 20:44:28.188642 16135 tablet_service.cc:1514] Processing DeleteTablet for tablet adbc98875ae147f4910e50ecc5dec292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:28 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:41982
I20260619 20:44:28.188735 16447 tablet_replica.cc:330] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: stopping tablet replica
I20260619 20:44:28.188723 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet adbc98875ae147f4910e50ecc5dec292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:28 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:50044
I20260619 20:44:28.188792 16447 raft_consensus.cc:2229] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:28.188803 16450 tablet_replica.cc:330] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: stopping tablet replica
I20260619 20:44:28.188855 16447 raft_consensus.cc:2258] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:28.188860 16450 raft_consensus.cc:2229] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:28.188899 16450 raft_consensus.cc:2258] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:28.188933 16448 tablet_replica.cc:330] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: stopping tablet replica
I20260619 20:44:28.188999 16448 raft_consensus.cc:2229] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:28.189036 16448 raft_consensus.cc:2258] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:28.189184 16447 ts_tablet_manager.cc:1905] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:28.189186 16450 ts_tablet_manager.cc:1905] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:28.189374 16448 ts_tablet_manager.cc:1905] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:28.190733 15854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:28.190956 15854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:28.191004 16450 ts_tablet_manager.cc:1918] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:28.191018 16447 ts_tablet_manager.cc:1918] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:28.191062 16447 log.cc:1196] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/adbc98875ae147f4910e50ecc5dec292
I20260619 20:44:28.191066 16450 log.cc:1196] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/adbc98875ae147f4910e50ecc5dec292
I20260619 20:44:28.191188 16448 ts_tablet_manager.cc:1918] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:28.191237 16448 log.cc:1196] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/adbc98875ae147f4910e50ecc5dec292
I20260619 20:44:28.191304 16447 ts_tablet_manager.cc:1939] T adbc98875ae147f4910e50ecc5dec292 P 498ceb5c83f54a548a6fe71714ccbac0: Deleting consensus metadata
I20260619 20:44:28.191308 16450 ts_tablet_manager.cc:1939] T adbc98875ae147f4910e50ecc5dec292 P 9ac309a295e142ddaaecd40b32453a93: Deleting consensus metadata
I20260619 20:44:28.191452 16448 ts_tablet_manager.cc:1939] T adbc98875ae147f4910e50ecc5dec292 P e3bb624ecf6b4059b231729c48e2529d: Deleting consensus metadata
I20260619 20:44:28.191603 15841 catalog_manager.cc:4841] TS 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377): tablet adbc98875ae147f4910e50ecc5dec292 (table test-table [id=a9fd3a11f8db4265b3b2eca65273ce1c]) successfully deleted
I20260619 20:44:28.191829 15838 catalog_manager.cc:4841] TS e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763): tablet adbc98875ae147f4910e50ecc5dec292 (table test-table [id=a9fd3a11f8db4265b3b2eca65273ce1c]) successfully deleted
I20260619 20:44:28.191902 15838 catalog_manager.cc:4841] TS 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035): tablet adbc98875ae147f4910e50ecc5dec292 (table test-table [id=a9fd3a11f8db4265b3b2eca65273ce1c]) successfully deleted
I20260619 20:44:28.192534 15993 tablet_service.cc:1467] Processing CreateTablet for tablet 432e36d73f544e4aa23bd876b3c28f80 (DEFAULT_TABLE table=test-table [id=b2eff9ee995d4c6e95939bcb2bbed81d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:28.192533 16135 tablet_service.cc:1467] Processing CreateTablet for tablet 432e36d73f544e4aa23bd876b3c28f80 (DEFAULT_TABLE table=test-table [id=b2eff9ee995d4c6e95939bcb2bbed81d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:28.192566 16276 tablet_service.cc:1467] Processing CreateTablet for tablet 432e36d73f544e4aa23bd876b3c28f80 (DEFAULT_TABLE table=test-table [id=b2eff9ee995d4c6e95939bcb2bbed81d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:28.192651 16276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432e36d73f544e4aa23bd876b3c28f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:28.192651 15993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432e36d73f544e4aa23bd876b3c28f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:28.192665 16135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432e36d73f544e4aa23bd876b3c28f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:28.193487 16459 tablet_bootstrap.cc:492] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: Bootstrap starting.
I20260619 20:44:28.193768 16459 tablet_bootstrap.cc:654] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:28.193888 16458 tablet_bootstrap.cc:492] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0: Bootstrap starting.
I20260619 20:44:28.194259 16458 tablet_bootstrap.cc:654] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:28.194415 16459 tablet_bootstrap.cc:492] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: No bootstrap required, opened a new log
I20260619 20:44:28.194504 16459 ts_tablet_manager.cc:1397] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:28.194571 16460 tablet_bootstrap.cc:492] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: Bootstrap starting.
I20260619 20:44:28.194679 16459 raft_consensus.cc:348] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.194748 16459 raft_consensus.cc:374] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:28.194748 16458 tablet_bootstrap.cc:492] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0: No bootstrap required, opened a new log
I20260619 20:44:28.194772 16459 raft_consensus.cc:729] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3bb624ecf6b4059b231729c48e2529d, State: Initialized, Role: FOLLOWER
I20260619 20:44:28.194785 16458 ts_tablet_manager.cc:1397] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:28.194823 16459 consensus_queue.cc:260] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.194876 16460 tablet_bootstrap.cc:654] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:28.194870 16458 raft_consensus.cc:348] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.194913 16458 raft_consensus.cc:374] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:28.194926 16458 raft_consensus.cc:729] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498ceb5c83f54a548a6fe71714ccbac0, State: Initialized, Role: FOLLOWER
I20260619 20:44:28.194926 16459 ts_tablet_manager.cc:1428] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:28.194988 16458 consensus_queue.cc:260] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.195081 16458 ts_tablet_manager.cc:1428] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:28.195739 16460 tablet_bootstrap.cc:492] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: No bootstrap required, opened a new log
I20260619 20:44:28.195859 16460 ts_tablet_manager.cc:1397] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:28.196029 16460 raft_consensus.cc:348] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.196101 16460 raft_consensus.cc:374] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:28.196116 16460 raft_consensus.cc:729] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac309a295e142ddaaecd40b32453a93, State: Initialized, Role: FOLLOWER
I20260619 20:44:28.196148 16460 consensus_queue.cc:260] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.196237 16460 ts_tablet_manager.cc:1428] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:28.343423 16442 raft_consensus.cc:482] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:28.343524 16442 raft_consensus.cc:504] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.343739 16442 leader_election.cc:288] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:28.343927 16014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432e36d73f544e4aa23bd876b3c28f80" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498ceb5c83f54a548a6fe71714ccbac0" is_pre_election: true
I20260619 20:44:28.343940 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432e36d73f544e4aa23bd876b3c28f80" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93" is_pre_election: true
I20260619 20:44:28.344033 16296 raft_consensus.cc:2454] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 0.
I20260619 20:44:28.344035 16014 raft_consensus.cc:2454] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 0.
I20260619 20:44:28.344241 16089 leader_election.cc:302] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, e3bb624ecf6b4059b231729c48e2529d; no voters:
I20260619 20:44:28.344378 16442 raft_consensus.cc:2781] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:28.344422 16442 raft_consensus.cc:482] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:28.344450 16442 raft_consensus.cc:3037] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:28.345151 16442 raft_consensus.cc:504] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.345291 16442 leader_election.cc:288] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 election: Requested vote from peers 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377), 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035)
I20260619 20:44:28.345427 16014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432e36d73f544e4aa23bd876b3c28f80" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498ceb5c83f54a548a6fe71714ccbac0"
I20260619 20:44:28.345463 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432e36d73f544e4aa23bd876b3c28f80" candidate_uuid: "e3bb624ecf6b4059b231729c48e2529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac309a295e142ddaaecd40b32453a93"
I20260619 20:44:28.345525 16014 raft_consensus.cc:3037] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:28.345531 16296 raft_consensus.cc:3037] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:28.346112 16014 raft_consensus.cc:2454] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 1.
I20260619 20:44:28.346252 16296 raft_consensus.cc:2454] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3bb624ecf6b4059b231729c48e2529d in term 1.
I20260619 20:44:28.346347 16089 leader_election.cc:302] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ceb5c83f54a548a6fe71714ccbac0, e3bb624ecf6b4059b231729c48e2529d; no voters:
I20260619 20:44:28.346462 16442 raft_consensus.cc:2781] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:28.346542 16442 raft_consensus.cc:686] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 1 LEADER]: Becoming Leader. State: Replica: e3bb624ecf6b4059b231729c48e2529d, State: Running, Role: LEADER
I20260619 20:44:28.346602 16442 consensus_queue.cc:237] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } }
I20260619 20:44:28.347183 15854 catalog_manager.cc:5495] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d reported cstate change: term changed from 0 to 1, leader changed from <none> to e3bb624ecf6b4059b231729c48e2529d (127.13.195.66). New cstate: current_term: 1 leader_uuid: "e3bb624ecf6b4059b231729c48e2529d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3bb624ecf6b4059b231729c48e2529d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 43763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:28.379850 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:28.377390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.13.195.66:43763)
0619 20:44:28.377564 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:28.377566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:28.377573 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:44:28.377591 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:28.377591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:28.377600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:28.377657 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:28.377837 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:28.378493 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:28.379204 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:28.379421 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:28.379479 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:28.379591 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:28.379617 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:28.379665 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:28.379712 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 20:44:28.379728 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":101,"threads_started":1}
I20260619 20:44:28.382046 16296 raft_consensus.cc:1261] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Refusing update from remote peer e3bb624ecf6b4059b231729c48e2529d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:28.382053 16014 raft_consensus.cc:1261] T 432e36d73f544e4aa23bd876b3c28f80 P 498ceb5c83f54a548a6fe71714ccbac0 [term 1 FOLLOWER]: Refusing update from remote peer e3bb624ecf6b4059b231729c48e2529d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:28.382277 16438 consensus_queue.cc:1035] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac309a295e142ddaaecd40b32453a93" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:28.382395 16442 consensus_queue.cc:1035] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [LEADER]: Connected to new peer: Peer: permanent_uuid: "498ceb5c83f54a548a6fe71714ccbac0" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:44:28.386797 16429 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 20:44:29.903159 15854 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:29.903259 15854 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:29.903999 15854 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fddb696c56e4f29ba44967f67ff76c9: Sending DeleteTablet for 3 replicas of tablet 432e36d73f544e4aa23bd876b3c28f80
I20260619 20:44:29.904232 16135 tablet_service.cc:1514] Processing DeleteTablet for tablet 432e36d73f544e4aa23bd876b3c28f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:29 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:41982
I20260619 20:44:29.904237 15993 tablet_service.cc:1514] Processing DeleteTablet for tablet 432e36d73f544e4aa23bd876b3c28f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:29 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:44264
I20260619 20:44:29.904289 16276 tablet_service.cc:1514] Processing DeleteTablet for tablet 432e36d73f544e4aa23bd876b3c28f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:29 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:50044
I20260619 20:44:29.904420 16501 tablet_replica.cc:330] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: stopping tablet replica
I20260619 20:44:29.904470 16501 raft_consensus.cc:2229] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:29.904547 16501 raft_consensus.cc:2258] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:29.904597 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 15928
I20260619 20:44:29.905400 16501 ts_tablet_manager.cc:1905] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:29.907372 16501 ts_tablet_manager.cc:1918] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:44:29.907379 16503 tablet_replica.cc:330] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: stopping tablet replica
I20260619 20:44:29.907428 16503 raft_consensus.cc:2229] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:29.907454 16503 raft_consensus.cc:2258] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:29.907455 16501 log.cc:1196] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/432e36d73f544e4aa23bd876b3c28f80
I20260619 20:44:29.907688 16501 ts_tablet_manager.cc:1939] T 432e36d73f544e4aa23bd876b3c28f80 P e3bb624ecf6b4059b231729c48e2529d: Deleting consensus metadata
I20260619 20:44:29.907688 16503 ts_tablet_manager.cc:1905] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:29.908109 15838 catalog_manager.cc:4841] TS e3bb624ecf6b4059b231729c48e2529d (127.13.195.66:43763): tablet 432e36d73f544e4aa23bd876b3c28f80 (table test-table [id=b2eff9ee995d4c6e95939bcb2bbed81d]) successfully deleted
I20260619 20:44:29.908967 16503 ts_tablet_manager.cc:1918] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:44:29.909021 16503 log.cc:1196] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/432e36d73f544e4aa23bd876b3c28f80
I20260619 20:44:29.909233 16503 ts_tablet_manager.cc:1939] T 432e36d73f544e4aa23bd876b3c28f80 P 9ac309a295e142ddaaecd40b32453a93: Deleting consensus metadata
I20260619 20:44:29.909544 15838 catalog_manager.cc:4841] TS 9ac309a295e142ddaaecd40b32453a93 (127.13.195.67:40035): tablet 432e36d73f544e4aa23bd876b3c28f80 (table test-table [id=b2eff9ee995d4c6e95939bcb2bbed81d]) successfully deleted
W20260619 20:44:29.912497 15841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:39377 (error 108)
I20260619 20:44:29.912739 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 16070
I20260619 20:44:29.914094 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:29.913894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39377 (local address 127.0.0.1:44296)
0619 20:44:29.914021 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:29.914062 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
W20260619 20:44:29.914213 15841 catalog_manager.cc:4584] TS 498ceb5c83f54a548a6fe71714ccbac0 (127.13.195.65:39377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 432e36d73f544e4aa23bd876b3c28f80: Network error: Client connection negotiation failed: client connection to 127.13.195.65:39377: connect: Connection refused (error 111)
I20260619 20:44:29.918812 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 16211
I20260619 20:44:29.923251 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 15822
2026-06-19T20:44:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5793 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:29 dist-test-slave-n2td krb5kdc[16508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:29 dist-test-slave-n2td krb5kdc[16508](info): set up 2 sockets
Jun 19 20:44:29 dist-test-slave-n2td krb5kdc[16508](info): commencing operation
krb5kdc: starting...
W20260619 20:44:30.965050 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:31.081224 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40923
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:40923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:31.164302 16521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:31.164539 16521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:31.164564 16521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:31.166378 16521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:31.166465 16521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:31.166484 16521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:31.166520 16521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:31.166553 16521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:31.168147 16521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:40923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:31.168439 16521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:31.168717 16521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:31.174664 16526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.174664 16529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.174698 16527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:31.174753 16521 server_base.cc:1034] running on GCE node
I20260619 20:44:31.175235 16521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:31.175506 16521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:31.176682 16521 hybrid_clock.cc:648] HybridClock initialized: now 1781901871176668 us; error 46 us; skew 500 ppm
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:31.179221 16521 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:31.179672 16521 webserver.cc:460] Webserver started at http://127.13.195.126:37947/ using document root <none> and password file <none>
I20260619 20:44:31.179915 16521 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:31.179983 16521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:31.180136 16521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:31.181190 16521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "8cf25de2169f483e85c622d64167ed88"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "130d6fd94339726f7e151eb69c717fa9"
server_key_iv: "19f2a158feb9ad90f1ed221f63a246f2"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.181545 16521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "8cf25de2169f483e85c622d64167ed88"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "130d6fd94339726f7e151eb69c717fa9"
server_key_iv: "19f2a158feb9ad90f1ed221f63a246f2"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.182943 16521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:31.183612 16536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.183780 16521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.183883 16521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "8cf25de2169f483e85c622d64167ed88"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "130d6fd94339726f7e151eb69c717fa9"
server_key_iv: "19f2a158feb9ad90f1ed221f63a246f2"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.183965 16521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:31.212090 16521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:31.214265 16521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:31.214511 16521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:31.218465 16521 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:40923
I20260619 20:44:31.218531 16588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:40923 every 8 connection(s)
I20260619 20:44:31.218904 16521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:31.219761 16589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:31.222081 16589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88: Bootstrap starting.
I20260619 20:44:31.222710 16589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:31.223136 16589 log.cc:824] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:31.223896 16589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88: No bootstrap required, opened a new log
I20260619 20:44:31.225525 16589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf25de2169f483e85c622d64167ed88" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40923 } }
I20260619 20:44:31.225667 16589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:31.225689 16589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf25de2169f483e85c622d64167ed88, State: Initialized, Role: FOLLOWER
I20260619 20:44:31.225863 16589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf25de2169f483e85c622d64167ed88" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40923 } }
I20260619 20:44:31.225950 16589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:31.225987 16589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:31.226034 16589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:31.226552 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 16521
I20260619 20:44:31.226651 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:31.226666 16589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf25de2169f483e85c622d64167ed88" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40923 } }
I20260619 20:44:31.226823 14093 external_mini_cluster.cc:1428] Setting key 392745f369135845543f349cb65b5583
I20260619 20:44:31.226797 16589 leader_election.cc:302] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cf25de2169f483e85c622d64167ed88; no voters:
I20260619 20:44:31.227005 16589 leader_election.cc:288] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:31.227025 16592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:31.227146 16592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [term 1 LEADER]: Becoming Leader. State: Replica: 8cf25de2169f483e85c622d64167ed88, State: Running, Role: LEADER
I20260619 20:44:31.227285 16589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:31.227293 16592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf25de2169f483e85c622d64167ed88" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40923 } }
I20260619 20:44:31.227937 16593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cf25de2169f483e85c622d64167ed88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf25de2169f483e85c622d64167ed88" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40923 } } }
I20260619 20:44:31.227959 16594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cf25de2169f483e85c622d64167ed88. Latest consensus state: current_term: 1 leader_uuid: "8cf25de2169f483e85c622d64167ed88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cf25de2169f483e85c622d64167ed88" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40923 } } }
I20260619 20:44:31.228089 16593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:31.228127 16594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:31.228430 16598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:31.229007 16598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:31.230620 16598 catalog_manager.cc:1347] Generated new cluster ID: ac6363db5953451ab113309bf0253643
I20260619 20:44:31.230674 16598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:31.239298 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:31.227440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.13.195.126:40923)
0619 20:44:31.227604 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:31.227615 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:31.227651 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:44:31.227765 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:31.227773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:31.227836 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:31.228864 (+ 1028us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:31.228981 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.230273 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.231348 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.231990 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.232421 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:31.235024 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:31.235029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:31.235029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:31.235043 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:31.237085 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.237588 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.237592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.237592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.237717 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.237928 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.237929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.237929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.238267 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:31.238851 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:31.238926 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:44:31.239075 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260619 20:44:31.243211 16598 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:31.243970 16598 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:31.251163 16598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cf25de2169f483e85c622d64167ed88: Generated new TSK 0
I20260619 20:44:31.251394 16598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:31.278990 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40923
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:31.360011 16615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:31.360260 16615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:31.360299 16615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:31.361470 16615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:31.361542 16615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:31.361588 16615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:31.363047 16615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:31.363318 16615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:31.363561 16615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:44:31.368712 16615 server_base.cc:1034] running on GCE node
W20260619 20:44:31.368683 16621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.368677 16623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.368683 16620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:31.369150 16615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:31.369410 16615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:31.370579 16615 hybrid_clock.cc:648] HybridClock initialized: now 1781901871370565 us; error 46 us; skew 500 ppm
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:31.373068 16615 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:31.373487 16615 webserver.cc:460] Webserver started at http://127.13.195.65:42587/ using document root <none> and password file <none>
I20260619 20:44:31.373854 16615 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:31.373914 16615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:31.374009 16615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:31.375166 16615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "388f5b925bda400abe0a3636e05e7860"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "08485d6046db06bdaff3ed2312ffac83"
server_key_iv: "96bb970b6966cc111a4b35db4c9abe8c"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.375487 16615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "388f5b925bda400abe0a3636e05e7860"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "08485d6046db06bdaff3ed2312ffac83"
server_key_iv: "96bb970b6966cc111a4b35db4c9abe8c"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.376756 16615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:31.377446 16631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.377630 16615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:31.377699 16615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "388f5b925bda400abe0a3636e05e7860"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "08485d6046db06bdaff3ed2312ffac83"
server_key_iv: "96bb970b6966cc111a4b35db4c9abe8c"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.377812 16615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:31.405043 16615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:31.407212 16615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:31.407374 16615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:31.407632 16615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:31.407955 16615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:31.407995 16615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.408020 16615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:31.408088 16615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.413861 16615 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:46611
I20260619 20:44:31.413930 16745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:46611 every 8 connection(s)
I20260619 20:44:31.414270 16615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:31.414391 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 16615
I20260619 20:44:31.414534 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:31.414727 14093 external_mini_cluster.cc:1428] Setting key 2262774a6cf12c9785d9c70938d586a9
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
I20260619 20:44:31.424507 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:31.415167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:47693 (local address 127.13.195.126:40923)
0619 20:44:31.415210 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:31.415213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:31.415424 (+ 211us) server_negotiation.cc:407] Connection header received
0619 20:44:31.415950 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:31.415952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:31.415964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:31.416043 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:31.417218 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.417716 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.418830 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.419051 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.419141 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:31.421466 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:31.421468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:31.421468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:31.421476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:31.423092 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.423508 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.423509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.423509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.423566 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.423848 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.423849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.423849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.423983 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:31.424061 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:31.424333 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 20:44:31.424413 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:31.424726 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:31.415078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40923 (local address 127.13.195.65:47693)
0619 20:44:31.415275 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:31.415322 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:31.415840 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:31.416098 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:31.416101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:31.416808 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:31.417111 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:31.417118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.417820 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.417821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:31.418732 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:31.418734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.419153 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.419154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:31.419277 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:31.419810 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:31.419818 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:31.421364 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:31.423180 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:31.423187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:31.423189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:31.423411 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:31.423634 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:31.423635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:31.423635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:31.423783 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:31.424052 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:31.424058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:31.424200 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 20:44:31.424312 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 20:44:31.424481 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
I20260619 20:44:31.425318 16746 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40923
I20260619 20:44:31.425420 16746 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:31.425622 16746 heartbeater.cc:511] Master 127.13.195.126:40923 requested a full tablet report, sending...
I20260619 20:44:31.426234 16553 ts_manager.cc:194] Registered new tserver with Master: 388f5b925bda400abe0a3636e05e7860 (127.13.195.65:46611)
I20260619 20:44:31.427106 16553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:47693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:31.439498 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40923
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:31.521196 16751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:31.521457 16751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:31.521497 16751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:31.522706 16751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:31.522783 16751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:31.522826 16751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:31.524299 16751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:31.524592 16751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:31.524832 16751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:31.530222 16757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.530222 16756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.530385 16759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:31.530491 16751 server_base.cc:1034] running on GCE node
I20260619 20:44:31.530720 16751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:31.530970 16751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:31.532140 16751 hybrid_clock.cc:648] HybridClock initialized: now 1781901871532126 us; error 48 us; skew 500 ppm
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:31.534723 16751 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:31.535173 16751 webserver.cc:460] Webserver started at http://127.13.195.66:35553/ using document root <none> and password file <none>
I20260619 20:44:31.535405 16751 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:31.535451 16751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:31.535606 16751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:31.536727 16751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "d6a4d268bba2471784cda6ef6e6e2b48"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "d5bdccdab13319b62268bfe81cc4624f"
server_key_iv: "c776b96db31c091bdf686f562abbdec3"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.537097 16751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "d6a4d268bba2471784cda6ef6e6e2b48"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "d5bdccdab13319b62268bfe81cc4624f"
server_key_iv: "c776b96db31c091bdf686f562abbdec3"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.538396 16751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:44:31.539163 16766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.539333 16751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:31.539410 16751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "d6a4d268bba2471784cda6ef6e6e2b48"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "d5bdccdab13319b62268bfe81cc4624f"
server_key_iv: "c776b96db31c091bdf686f562abbdec3"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.539469 16751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:31.554548 16751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:31.556460 16751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:31.556571 16751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:31.556751 16751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:31.557062 16751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:31.557096 16751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.557155 16751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:31.557199 16751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.562975 16751 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:39105
I20260619 20:44:31.563084 16880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:39105 every 8 connection(s)
I20260619 20:44:31.563380 16751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:31.563948 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 16751
I20260619 20:44:31.564056 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:31.564267 14093 external_mini_cluster.cc:1428] Setting key ff97e6f09b19339c084295c236ee4865
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:44:31.574227 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:31.564151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:38323 (local address 127.13.195.126:40923)
0619 20:44:31.564477 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:31.564479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:31.564492 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:44:31.565134 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:31.565135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:31.565148 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:31.565202 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:31.566549 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.567352 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.568361 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.568712 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.568794 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:31.571242 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:31.571244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:31.571244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:31.571252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:31.572852 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.573308 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.573309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.573310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.573375 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.573607 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.573608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.573608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.573714 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:31.573766 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:31.574084 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 20:44:31.574160 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260619 20:44:31.574527 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:31.564123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40923 (local address 127.13.195.66:38323)
0619 20:44:31.564327 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:31.564378 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:31.565007 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:31.565291 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:31.565296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:31.566133 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:31.566425 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:31.566468 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.567448 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.567449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:31.568258 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:31.568260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.568801 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.568801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:31.568902 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:31.569484 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:31.569492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:31.571138 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:31.572933 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:31.572939 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:31.572940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:31.573236 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:31.573447 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:31.573448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:31.573448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:31.573547 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:31.573768 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:31.573773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:31.573923 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 20:44:31.574057 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 20:44:31.574238 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
I20260619 20:44:31.575133 16881 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40923
I20260619 20:44:31.575242 16881 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:31.575425 16881 heartbeater.cc:511] Master 127.13.195.126:40923 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
I20260619 20:44:31.575835 16553 ts_manager.cc:194] Registered new tserver with Master: d6a4d268bba2471784cda6ef6e6e2b48 (127.13.195.66:39105)
I20260619 20:44:31.576258 16553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:38323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:31.589360 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40923
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:31.675858 16886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:31.676080 16886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:31.676103 16886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:31.677278 16886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:31.677326 16886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:31.677353 16886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:31.678917 16886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:31.679347 16886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:31.679616 16886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:31.685817 16891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.685837 16892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:31.685837 16894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:31.686237 16886 server_base.cc:1034] running on GCE node
I20260619 20:44:31.686472 16886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:31.686769 16886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:31.687908 16886 hybrid_clock.cc:648] HybridClock initialized: now 1781901871687889 us; error 35 us; skew 500 ppm
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:31.690382 16886 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:31.690833 16886 webserver.cc:460] Webserver started at http://127.13.195.67:43869/ using document root <none> and password file <none>
I20260619 20:44:31.691066 16886 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:31.691134 16886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:31.691287 16886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:31.692363 16886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "90638d48fd384068b4c66ff71e7c008d"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "c779db34127a31e468961877a5b5adc3"
server_key_iv: "f093f30bca21f02a80620b6ee1557937"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.692726 16886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "90638d48fd384068b4c66ff71e7c008d"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "c779db34127a31e468961877a5b5adc3"
server_key_iv: "f093f30bca21f02a80620b6ee1557937"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.694044 16886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:44:31.694813 16901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.694979 16886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.695079 16886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "90638d48fd384068b4c66ff71e7c008d"
format_stamp: "Formatted at 2026-06-19 20:44:31 on dist-test-slave-n2td"
server_key: "c779db34127a31e468961877a5b5adc3"
server_key_iv: "f093f30bca21f02a80620b6ee1557937"
server_key_version: "encryptionkey@0"
I20260619 20:44:31.695149 16886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:31.710251 16886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:31.712668 16886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:31.712838 16886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:31.713083 16886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:31.713416 16886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:31.713476 16886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.713518 16886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:31.713543 16886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:31.719483 16886 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:46267
I20260619 20:44:31.719532 17015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:46267 every 8 connection(s)
I20260619 20:44:31.719897 16886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:31.724035 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 16886
I20260619 20:44:31.724169 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:31.724376 14093 external_mini_cluster.cc:1428] Setting key ed53f11e38501bce42bc325d8f9f87e9
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[16508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901871, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:31.729696 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:31.720527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:36843 (local address 127.13.195.126:40923)
0619 20:44:31.720604 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:31.720606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:31.720847 (+ 241us) server_negotiation.cc:407] Connection header received
0619 20:44:31.721387 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:31.721388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:31.721401 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:31.721486 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:31.722412 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.722937 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.724035 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.724322 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.724413 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:31.726818 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:31.726820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:31.726820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:31.726828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:31.728272 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.728693 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.728694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.728694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.728754 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:31.729021 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:31.729022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:31.729023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:31.729147 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:31.729228 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:31.729522 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 20:44:31.729605 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:44:31.729899 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:31.720532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40923 (local address 127.13.195.67:36843)
0619 20:44:31.720702 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:31.720738 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:31.721277 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:31.721563 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:31.721567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:31.722119 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:31.722308 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:31.722313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.723054 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.723055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:31.723931 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:31.723934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:31.724422 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:31.724423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:31.724548 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:31.725072 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:31.725102 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:31.726723 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:31.728355 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:31.728361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:31.728363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:31.728602 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:31.728831 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:31.728832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:31.728833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:31.728952 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:31.729218 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:31.729223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:31.729378 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 20:44:31.729500 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 20:44:31.729657 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260619 20:44:31.730383 17016 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40923
I20260619 20:44:31.730563 17016 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:31.730772 17016 heartbeater.cc:511] Master 127.13.195.126:40923 requested a full tablet report, sending...
I20260619 20:44:31.731191 16553 ts_manager.cc:194] Registered new tserver with Master: 90638d48fd384068b4c66ff71e7c008d (127.13.195.67:46267)
I20260619 20:44:31.731571 16553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:36843
I20260619 20:44:31.736330 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:44:31.747898 16596 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:31.746636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55036 (local address 127.13.195.126:40923)
0619 20:44:31.746690 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:31.746707 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:31.746889 (+ 182us) server_negotiation.cc:407] Connection header received
0619 20:44:31.746989 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:31.746990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:31.747005 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:31.747084 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:31.747810 (+ 726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55036: BlockingRecv error: recv got EOF from 127.0.0.1:55036 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:44:31.748831 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 16615
I20260619 20:44:31.758010 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 16751
I20260619 20:44:31.765498 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 16886
I20260619 20:44:31.771437 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 16521
2026-06-19T20:44:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1848 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[17028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[17028](info): set up 2 sockets
Jun 19 20:44:31 dist-test-slave-n2td krb5kdc[17028](info): commencing operation
krb5kdc: starting...
W20260619 20:44:32.913190 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:32 dist-test-slave-n2td krb5kdc[17028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:33.032850 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36645
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:36645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:33.113284 17041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:33.113512 17041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:33.113536 17041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:33.114711 17041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:33.114748 17041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:33.114763 17041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:33.114775 17041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:33.114787 17041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:33.116279 17041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:36645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:33.116479 17041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:33.116652 17041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:33.121578 17049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:33.121588 17046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:33.121788 17047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:33.121986 17041 server_base.cc:1034] running on GCE node
I20260619 20:44:33.122282 17041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:33.122617 17041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:33.123762 17041 hybrid_clock.cc:648] HybridClock initialized: now 1781901873123741 us; error 40 us; skew 500 ppm
Jun 19 20:44:33 dist-test-slave-n2td krb5kdc[17028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901873, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:33.126590 17041 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:33.127104 17041 webserver.cc:460] Webserver started at http://127.13.195.126:32813/ using document root <none> and password file <none>
I20260619 20:44:33.127348 17041 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:33.127393 17041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:33.127497 17041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:33.128579 17041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "f5dc666165b14a18a5e10e2815d836c2"
format_stamp: "Formatted at 2026-06-19 20:44:33 on dist-test-slave-n2td"
server_key: "9dda0fde06c6e4c1881c786761f20281"
server_key_iv: "9a620be9c8341e2f503864c589d7d25c"
server_key_version: "encryptionkey@0"
I20260619 20:44:33.128892 17041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "f5dc666165b14a18a5e10e2815d836c2"
format_stamp: "Formatted at 2026-06-19 20:44:33 on dist-test-slave-n2td"
server_key: "9dda0fde06c6e4c1881c786761f20281"
server_key_iv: "9a620be9c8341e2f503864c589d7d25c"
server_key_version: "encryptionkey@0"
I20260619 20:44:33.130304 17041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:33.131168 17057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:33.131367 17041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:33.131471 17041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "f5dc666165b14a18a5e10e2815d836c2"
format_stamp: "Formatted at 2026-06-19 20:44:33 on dist-test-slave-n2td"
server_key: "9dda0fde06c6e4c1881c786761f20281"
server_key_iv: "9a620be9c8341e2f503864c589d7d25c"
server_key_version: "encryptionkey@0"
I20260619 20:44:33.131554 17041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:33.149734 17041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:33.152323 17041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:33.152487 17041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:33.156800 17041 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:36645
I20260619 20:44:33.156831 17109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:36645 every 8 connection(s)
I20260619 20:44:33.157341 17041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:33.157722 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17041
I20260619 20:44:33.157837 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:33.158010 14093 external_mini_cluster.cc:1428] Setting key b7f025f42cecceeba236524d4bd828ab
I20260619 20:44:33.158176 17110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:33.161599 17110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2: Bootstrap starting.
I20260619 20:44:33.162326 17110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:33.162672 17110 log.cc:824] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:33.163389 17110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2: No bootstrap required, opened a new log
Jun 19 20:44:33 dist-test-slave-n2td krb5kdc[17028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:33.166003 17110 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5dc666165b14a18a5e10e2815d836c2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36645 } }
I20260619 20:44:33.166205 17110 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:33.166251 17110 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5dc666165b14a18a5e10e2815d836c2, State: Initialized, Role: FOLLOWER
I20260619 20:44:33.166401 17110 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5dc666165b14a18a5e10e2815d836c2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36645 } }
I20260619 20:44:33.166539 17110 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:33.166568 17110 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:33.166637 17110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:33.167390 17110 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5dc666165b14a18a5e10e2815d836c2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36645 } }
I20260619 20:44:33.167507 17110 leader_election.cc:302] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5dc666165b14a18a5e10e2815d836c2; no voters:
I20260619 20:44:33.167706 17110 leader_election.cc:288] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:33.167737 17115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:33.167932 17115 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [term 1 LEADER]: Becoming Leader. State: Replica: f5dc666165b14a18a5e10e2815d836c2, State: Running, Role: LEADER
I20260619 20:44:33.168025 17110 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:33.168051 17115 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5dc666165b14a18a5e10e2815d836c2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36645 } }
I20260619 20:44:33.168475 17115 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5dc666165b14a18a5e10e2815d836c2. Latest consensus state: current_term: 1 leader_uuid: "f5dc666165b14a18a5e10e2815d836c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5dc666165b14a18a5e10e2815d836c2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36645 } } }
I20260619 20:44:33.168560 17115 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:33.168745 17117 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5dc666165b14a18a5e10e2815d836c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5dc666165b14a18a5e10e2815d836c2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36645 } } }
I20260619 20:44:33.168840 17117 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:33.168975 17122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:33.169555 17122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:33.170047 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:33.158555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.13.195.126:36645)
0619 20:44:33.158730 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:33.158740 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:33.158772 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:44:33.159320 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:33.159327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:33.159375 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:33.159491 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:33.160325 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:33.161252 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:33.162296 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:33.162886 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:33.163016 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:33.165635 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:33.165640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:33.165640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:33.165656 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:33.167740 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:33.168227 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:33.168231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:33.168231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:33.168341 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:33.168586 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:33.168587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:33.168587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:33.168939 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:33.169612 (+ 673us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:33.169695 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 20:44:33.169842 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260619 20:44:33.170931 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:44:33.171396 17122 catalog_manager.cc:1347] Generated new cluster ID: 90b30c8e9ffb495286c38ed0edf82882
I20260619 20:44:33.171447 17122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:33.176043 17122 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:33.176884 17122 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:33.182313 17122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5dc666165b14a18a5e10e2815d836c2: Generated new TSK 0
I20260619 20:44:33.182561 17122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:44:33.193701 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:33.189695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37428 (local address 127.13.195.126:36645)
0619 20:44:33.189750 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:33.189752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:33.189865 (+ 113us) server_negotiation.cc:407] Connection header received
0619 20:44:33.189913 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:33.189914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:33.189925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:33.190003 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:33.190362 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:33.191194 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:33.191951 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:33.192298 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:33.192374 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:33.193118 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:33.193119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:33.193120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:33.193370 (+ 250us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:33.193391 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:33.193399 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:33.193480 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:33.193581 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:44:33.193610 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:44:33.196337 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17041
2026-06-19T20:44:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1421 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:33 dist-test-slave-n2td krb5kdc[17143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:33 dist-test-slave-n2td krb5kdc[17143](info): set up 2 sockets
Jun 19 20:44:33 dist-test-slave-n2td krb5kdc[17143](info): commencing operation
krb5kdc: starting...
W20260619 20:44:34.241307 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:34.361451 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:33575
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:33575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:34.442395 17156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:34.442663 17156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:34.442703 17156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:34.443853 17156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:34.443931 17156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:34.443954 17156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:34.443974 17156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:34.444005 17156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:34.445616 17156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:33575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:33575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:34.445927 17156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:34.446171 17156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:34.451157 17164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:34.451167 17162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:34.451174 17161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:34.451412 17156 server_base.cc:1034] running on GCE node
I20260619 20:44:34.451710 17156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:34.451998 17156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:34.453166 17156 hybrid_clock.cc:648] HybridClock initialized: now 1781901874453143 us; error 35 us; skew 500 ppm
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:34.455721 17156 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:34.456131 17156 webserver.cc:460] Webserver started at http://127.13.195.126:39033/ using document root <none> and password file <none>
I20260619 20:44:34.456358 17156 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:34.456427 17156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:34.456583 17156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:34.457638 17156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "874669f34f8a43539c973300ef4e727f"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "74a7e3877f6a690f3358a03b3522840d"
server_key_iv: "901f322519c92ffb8f94c9ceda4daf8a"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.458010 17156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "874669f34f8a43539c973300ef4e727f"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "74a7e3877f6a690f3358a03b3522840d"
server_key_iv: "901f322519c92ffb8f94c9ceda4daf8a"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.459528 17156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:34.460350 17171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.460561 17156 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:34.460675 17156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "874669f34f8a43539c973300ef4e727f"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "74a7e3877f6a690f3358a03b3522840d"
server_key_iv: "901f322519c92ffb8f94c9ceda4daf8a"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.460764 17156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:34.492806 17156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:34.494951 17156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:34.495119 17156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:34.499636 17156 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:33575
I20260619 20:44:34.499668 17224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:33575 every 8 connection(s)
I20260619 20:44:34.500041 17156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:34.501112 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:34.503895 17225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f: Bootstrap starting.
I20260619 20:44:34.504649 17225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:34.504988 17225 log.cc:824] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:34.505791 17225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f: No bootstrap required, opened a new log
I20260619 20:44:34.506491 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17156
I20260619 20:44:34.506618 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:34.506783 14093 external_mini_cluster.cc:1428] Setting key 5e8dc9ad5540432519728a111f08ae27
I20260619 20:44:34.508602 17225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874669f34f8a43539c973300ef4e727f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 33575 } }
I20260619 20:44:34.508777 17225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:34.508816 17225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874669f34f8a43539c973300ef4e727f, State: Initialized, Role: FOLLOWER
I20260619 20:44:34.508955 17225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874669f34f8a43539c973300ef4e727f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 33575 } }
I20260619 20:44:34.509032 17225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:34.509055 17225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:34.509125 17225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:34.509758 17225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874669f34f8a43539c973300ef4e727f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 33575 } }
I20260619 20:44:34.509871 17225 leader_election.cc:302] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 874669f34f8a43539c973300ef4e727f; no voters:
I20260619 20:44:34.510020 17225 leader_election.cc:288] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:34.510090 17230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:34.510249 17225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:34.510273 17230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [term 1 LEADER]: Becoming Leader. State: Replica: 874669f34f8a43539c973300ef4e727f, State: Running, Role: LEADER
I20260619 20:44:34.510355 17230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874669f34f8a43539c973300ef4e727f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 33575 } }
I20260619 20:44:34.510682 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "874669f34f8a43539c973300ef4e727f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874669f34f8a43539c973300ef4e727f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 33575 } } }
I20260619 20:44:34.510773 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:34.510671 17230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874669f34f8a43539c973300ef4e727f. Latest consensus state: current_term: 1 leader_uuid: "874669f34f8a43539c973300ef4e727f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874669f34f8a43539c973300ef4e727f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 33575 } } }
I20260619 20:44:34.510967 17230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:34.511363 17238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:34.511920 17238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:34.514158 17238 catalog_manager.cc:1347] Generated new cluster ID: e99b9360434d42269cc338ee7896f459
I20260619 20:44:34.514211 17238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:34.518651 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:34.507267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.13.195.126:33575)
0619 20:44:34.507499 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:34.507509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:34.507543 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:44:34.508550 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:34.508559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:34.508624 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:34.508778 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:34.509516 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.510966 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.512058 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.512657 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.512766 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:34.515415 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:34.515419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:34.515419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:34.515431 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:34.517118 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:34.517516 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:34.517519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:34.517520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:34.517607 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:34.517813 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:34.517814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:34.517814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:34.518054 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:34.518228 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:34.518322 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:44:34.518488 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":123,"threads_started":1}
I20260619 20:44:34.529970 17238 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:34.530594 17238 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:34.540526 17238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 874669f34f8a43539c973300ef4e727f: Generated new TSK 0
I20260619 20:44:34.540726 17238 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:34.573385 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:33575
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:34.663729 17251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:34.663950 17251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:34.663973 17251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:34.665161 17251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:34.665203 17251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:34.665230 17251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:34.666752 17251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:33575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:34.667001 17251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:34.667263 17251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:34.672351 17259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:34.672422 17257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:34.672648 17256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:34.672621 17251 server_base.cc:1034] running on GCE node
I20260619 20:44:34.672878 17251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:34.673080 17251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:34.674263 17251 hybrid_clock.cc:648] HybridClock initialized: now 1781901874674236 us; error 44 us; skew 500 ppm
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:34.676867 17251 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:34.677326 17251 webserver.cc:460] Webserver started at http://127.13.195.65:37135/ using document root <none> and password file <none>
I20260619 20:44:34.677512 17251 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:34.677551 17251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:34.677641 17251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:34.678783 17251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "6a4d47ac29254c609b0c12325c865b9f"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "f869842ce2f2d0534c700ff155f004f2"
server_key_iv: "040ecc9ea41568775413d56196bce5ea"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.679112 17251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "6a4d47ac29254c609b0c12325c865b9f"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "f869842ce2f2d0534c700ff155f004f2"
server_key_iv: "040ecc9ea41568775413d56196bce5ea"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.680428 17251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:34.681169 17266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.681422 17251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:34.681497 17251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "6a4d47ac29254c609b0c12325c865b9f"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "f869842ce2f2d0534c700ff155f004f2"
server_key_iv: "040ecc9ea41568775413d56196bce5ea"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.681589 17251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:34.704022 17251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:34.706504 17251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:34.706676 17251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:34.706933 17251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:34.707247 17251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:34.707280 17251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.707325 17251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:34.707352 17251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.713730 17251 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:39761
I20260619 20:44:34.713774 17384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:39761 every 8 connection(s)
I20260619 20:44:34.714133 17251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:34.719130 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17251
I20260619 20:44:34.719254 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:34.719460 14093 external_mini_cluster.cc:1428] Setting key d243ae06c8d8fa79665a25db7fda2ed8
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:34.724303 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:34.715019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:39445 (local address 127.13.195.126:33575)
0619 20:44:34.715079 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:34.715082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:34.715316 (+ 234us) server_negotiation.cc:407] Connection header received
0619 20:44:34.715739 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:34.715741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:34.715752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:34.715815 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:34.716723 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.717232 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.718275 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.718547 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.718602 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:34.721003 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:34.721005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:34.721005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:34.721013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:34.722657 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:34.723180 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:34.723181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:34.723182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:34.723259 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:34.723572 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:34.723573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:34.723573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:34.723712 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:34.723755 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:34.724120 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 20:44:34.724212 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:44:34.724494 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:34.714917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:33575 (local address 127.13.195.65:39445)
0619 20:44:34.715139 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:34.715182 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:34.715656 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:34.715871 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:34.715875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:34.716431 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:34.716631 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:34.716635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.717341 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.717343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:34.718182 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:34.718185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.718620 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.718621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:34.718726 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:34.719335 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:34.719345 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:34.720912 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:34.722763 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:34.722771 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:34.722773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:34.723104 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:34.723341 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:34.723342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:34.723342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:34.723491 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:34.723806 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:34.723814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:34.723968 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 20:44:34.724085 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 20:44:34.724247 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260619 20:44:34.724942 17385 heartbeater.cc:347] Connected to a master server at 127.13.195.126:33575
I20260619 20:44:34.725047 17385 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:34.725253 17385 heartbeater.cc:511] Master 127.13.195.126:33575 requested a full tablet report, sending...
I20260619 20:44:34.725768 17188 ts_manager.cc:194] Registered new tserver with Master: 6a4d47ac29254c609b0c12325c865b9f (127.13.195.65:39761)
I20260619 20:44:34.726378 17188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:39445
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:34.745961 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:33575
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:34.828187 17390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:34.828410 17390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:34.828436 17390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:34.829583 17390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:34.829628 17390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:34.829656 17390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:34.831214 17390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:33575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:34.831425 17390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:34.831609 17390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:34.836990 17395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:34.837030 17390 server_base.cc:1034] running on GCE node
W20260619 20:44:34.836990 17398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:34.836975 17396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:34.837425 17390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:34.837697 17390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:34.838884 17390 hybrid_clock.cc:648] HybridClock initialized: now 1781901874838882 us; error 44 us; skew 500 ppm
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:34.841508 17390 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:34.841935 17390 webserver.cc:460] Webserver started at http://127.13.195.66:45217/ using document root <none> and password file <none>
I20260619 20:44:34.842171 17390 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:34.842242 17390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:34.842392 17390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:34.843467 17390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "affbfff94cfd47608325f08451989e69"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "1cf6c735ced96e454a31d851c0545018"
server_key_iv: "0ca385cd68dbd9cd4b848bfb9f406197"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.843847 17390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "affbfff94cfd47608325f08451989e69"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "1cf6c735ced96e454a31d851c0545018"
server_key_iv: "0ca385cd68dbd9cd4b848bfb9f406197"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.845269 17390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:34.846045 17405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.846293 17390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:34.846402 17390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "affbfff94cfd47608325f08451989e69"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "1cf6c735ced96e454a31d851c0545018"
server_key_iv: "0ca385cd68dbd9cd4b848bfb9f406197"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.846536 17390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:34.857700 17390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:34.859768 17390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:34.859988 17390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:34.860224 17390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:34.860535 17390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:34.860590 17390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.860625 17390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:34.860656 17390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.867074 17390 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:38681
I20260619 20:44:34.867130 17519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:38681 every 8 connection(s)
I20260619 20:44:34.867491 17390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:34.870934 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17390
I20260619 20:44:34.871023 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:34.871229 14093 external_mini_cluster.cc:1428] Setting key 36dced1fe4f3446f601bf27bea7e7a32
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:34.877830 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:34.868290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:52833 (local address 127.13.195.126:33575)
0619 20:44:34.868374 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:34.868377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:34.868586 (+ 209us) server_negotiation.cc:407] Connection header received
0619 20:44:34.869188 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:34.869189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:34.869198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:34.869263 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:34.870171 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.870686 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.872117 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.872420 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.872502 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:34.874946 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:34.874948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:34.874948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:34.874956 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:34.876376 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:34.876824 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:34.876825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:34.876826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:34.876886 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:34.877150 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:34.877151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:34.877152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:34.877271 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:34.877347 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:34.877652 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 20:44:34.877737 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 20:44:34.878031 17522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:34.868281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:33575 (local address 127.13.195.66:52833)
0619 20:44:34.868453 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:34.868497 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:34.869097 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:34.869320 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:34.869323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:34.869880 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:34.870087 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:34.870092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.870806 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.870807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:34.872022 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:34.872026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:34.872513 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:34.872514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:34.872617 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:34.873170 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:34.873178 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:34.874838 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:34.876483 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:34.876491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:34.876493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:34.876731 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:34.876964 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:34.876965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:34.876965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:34.877081 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:34.877339 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:34.877346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:34.877508 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 20:44:34.877617 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:44:34.877784 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260619 20:44:34.878512 17520 heartbeater.cc:347] Connected to a master server at 127.13.195.126:33575
I20260619 20:44:34.878610 17520 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:34.878821 17520 heartbeater.cc:511] Master 127.13.195.126:33575 requested a full tablet report, sending...
I20260619 20:44:34.879217 17188 ts_manager.cc:194] Registered new tserver with Master: affbfff94cfd47608325f08451989e69 (127.13.195.66:38681)
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:44:34.879662 17188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:52833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:34.898228 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:33575
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:34.980026 17525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:34.980237 17525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:34.980259 17525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:34.981323 17525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:34.981361 17525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:34.981387 17525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:34.982882 17525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:33575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:34.983078 17525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:34.983263 17525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:34.988368 17530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:34.988412 17531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:34.988446 17533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:34.988660 17525 server_base.cc:1034] running on GCE node
I20260619 20:44:34.988853 17525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:34.989053 17525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:34.990190 17525 hybrid_clock.cc:648] HybridClock initialized: now 1781901874990175 us; error 28 us; skew 500 ppm
Jun 19 20:44:34 dist-test-slave-n2td krb5kdc[17143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:34.992686 17525 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:34.993175 17525 webserver.cc:460] Webserver started at http://127.13.195.67:34999/ using document root <none> and password file <none>
I20260619 20:44:34.993355 17525 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:34.993392 17525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:34.993480 17525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:34.994532 17525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "dcda27faa00e4a1cb8d9204b8077fcb4"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "d6e0f62e82aacf60e0c1800f5ec60b0b"
server_key_iv: "f591a93e5bae30a746e94cf05f8220ca"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.994839 17525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "dcda27faa00e4a1cb8d9204b8077fcb4"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "d6e0f62e82aacf60e0c1800f5ec60b0b"
server_key_iv: "f591a93e5bae30a746e94cf05f8220ca"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.996078 17525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:34.996807 17540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:34.996989 17525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:34.997071 17525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "dcda27faa00e4a1cb8d9204b8077fcb4"
format_stamp: "Formatted at 2026-06-19 20:44:34 on dist-test-slave-n2td"
server_key: "d6e0f62e82aacf60e0c1800f5ec60b0b"
server_key_iv: "f591a93e5bae30a746e94cf05f8220ca"
server_key_version: "encryptionkey@0"
I20260619 20:44:34.997123 17525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:35.030030 17525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:35.032550 17525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:35.032673 17525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:35.032863 17525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:35.033253 17525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:35.033316 17525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:35.033363 17525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:35.033391 17525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:35.039044 17525 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:36241
I20260619 20:44:35.039084 17654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:36241 every 8 connection(s)
I20260619 20:44:35.039423 17525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:35.043578 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17525
I20260619 20:44:35.043654 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:35.043834 14093 external_mini_cluster.cc:1428] Setting key fccadc04a880e54acaebaa2574ec2121
Jun 19 20:44:35 dist-test-slave-n2td krb5kdc[17143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901874, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:35.048727 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:35.040140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:44295 (local address 127.13.195.126:33575)
0619 20:44:35.040222 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:35.040225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:35.040378 (+ 153us) server_negotiation.cc:407] Connection header received
0619 20:44:35.040959 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:35.040960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:35.040969 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:35.041019 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:35.041941 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:35.042507 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:35.043474 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:35.043691 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:35.043774 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:35.046107 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:35.046109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:35.046109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:35.046121 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:35.047350 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:35.047778 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:35.047779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:35.047779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:35.047837 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:35.048085 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:35.048086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:35.048086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:35.048201 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:35.048277 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:35.048555 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 20:44:35.048633 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 20:44:35.048924 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:35.040097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:33575 (local address 127.13.195.67:44295)
0619 20:44:35.040273 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:35.040310 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:35.040849 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:35.041096 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:35.041101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:35.041677 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:35.041869 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:35.041873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:35.042579 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:35.042579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:35.043367 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:35.043368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:35.043779 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:35.043780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:35.043928 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:35.044550 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:35.044561 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:35.046001 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:35.047459 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:35.047465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:35.047467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:35.047684 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:35.047905 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:35.047905 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:35.047906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:35.048014 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:35.048269 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:35.048275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:35.048418 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:44:35.048534 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 20:44:35.048687 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260619 20:44:35.049355 17655 heartbeater.cc:347] Connected to a master server at 127.13.195.126:33575
I20260619 20:44:35.049508 17655 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:35.049698 17655 heartbeater.cc:511] Master 127.13.195.126:33575 requested a full tablet report, sending...
I20260619 20:44:35.050072 17188 ts_manager.cc:194] Registered new tserver with Master: dcda27faa00e4a1cb8d9204b8077fcb4 (127.13.195.67:36241)
I20260619 20:44:35.050403 17188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:44295
I20260619 20:44:35.055548 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:44:35 dist-test-slave-n2td krb5kdc[17143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901875, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 20:44:35 dist-test-slave-n2td krb5kdc[17143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901875, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:35.077621 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:35.070610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.13.195.126:33575)
0619 20:44:35.070674 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:35.070676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:35.070832 (+ 156us) server_negotiation.cc:407] Connection header received
0619 20:44:35.070881 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:35.070882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:35.070891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:35.070951 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:35.071494 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:35.071977 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:35.072700 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:35.072876 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:35.072926 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:35.075167 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:35.075168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:35.075169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:35.075175 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:35.076577 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:35.076876 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:35.076877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:35.076878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:35.076923 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:35.077132 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:35.077133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:35.077133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:35.077272 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:35.077345 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:35.077492 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 20:44:35.077553 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 20:44:35.077894 17188 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59708
I20260619 20:44:35.078338 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17251
I20260619 20:44:35.083087 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17390
I20260619 20:44:35.087270 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17525
I20260619 20:44:35.091660 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17156
2026-06-19T20:44:35Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1897 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:35 dist-test-slave-n2td krb5kdc[17667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:35 dist-test-slave-n2td krb5kdc[17667](info): set up 2 sockets
Jun 19 20:44:35 dist-test-slave-n2td krb5kdc[17667](info): commencing operation
krb5kdc: starting...
W20260619 20:44:36.133956 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:36.253196 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:34127
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:34127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:36.336719 17680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:36.337020 17680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:36.337069 17680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:36.338387 17680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:36.338483 17680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:36.338523 17680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:36.338550 17680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:36.338577 17680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:36.340137 17680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:34127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:34127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:36.340423 17680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:36.340658 17680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:36.346175 17685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.346124 17686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.346102 17688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:36.346102 17680 server_base.cc:1034] running on GCE node
I20260619 20:44:36.346601 17680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:36.346907 17680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:36.348110 17680 hybrid_clock.cc:648] HybridClock initialized: now 1781901876348080 us; error 48 us; skew 500 ppm
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:36.350677 17680 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:36.351151 17680 webserver.cc:460] Webserver started at http://127.13.195.126:38201/ using document root <none> and password file <none>
I20260619 20:44:36.351384 17680 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:36.351451 17680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:36.351610 17680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:36.352712 17680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "6934034d51fc4098bce1d56a7cf0e332"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "ed52c89d6a6db4e11128639126e3ac7e"
server_key_iv: "f3427f03834d91da07f9c86affbd9287"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.353082 17680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "6934034d51fc4098bce1d56a7cf0e332"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "ed52c89d6a6db4e11128639126e3ac7e"
server_key_iv: "f3427f03834d91da07f9c86affbd9287"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.354540 17680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:44:36.355309 17695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.355510 17680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:36.355609 17680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "6934034d51fc4098bce1d56a7cf0e332"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "ed52c89d6a6db4e11128639126e3ac7e"
server_key_iv: "f3427f03834d91da07f9c86affbd9287"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.355697 17680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:36.372805 17680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:36.375231 17680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:36.375404 17680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:36.379459 17680 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:34127
I20260619 20:44:36.379499 17748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:34127 every 8 connection(s)
I20260619 20:44:36.380026 17680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:36.380729 17749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:36.383109 17749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332: Bootstrap starting.
I20260619 20:44:36.383648 17749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:36.383909 17749 log.cc:824] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:36.384565 17749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332: No bootstrap required, opened a new log
I20260619 20:44:36.386072 17749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6934034d51fc4098bce1d56a7cf0e332" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34127 } }
I20260619 20:44:36.386204 17749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:36.386226 17749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6934034d51fc4098bce1d56a7cf0e332, State: Initialized, Role: FOLLOWER
I20260619 20:44:36.386323 17749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6934034d51fc4098bce1d56a7cf0e332" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34127 } }
I20260619 20:44:36.386411 17749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:36.386447 17749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:36.386541 17749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:36.387420 17749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6934034d51fc4098bce1d56a7cf0e332" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34127 } }
I20260619 20:44:36.387540 17749 leader_election.cc:302] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6934034d51fc4098bce1d56a7cf0e332; no voters:
I20260619 20:44:36.387722 17749 leader_election.cc:288] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:36.387778 17752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:36.387955 17752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [term 1 LEADER]: Becoming Leader. State: Replica: 6934034d51fc4098bce1d56a7cf0e332, State: Running, Role: LEADER
I20260619 20:44:36.388187 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17680
I20260619 20:44:36.388284 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:36.388427 14093 external_mini_cluster.cc:1428] Setting key c778e2b740479ecb3b0249bb0cc98654
I20260619 20:44:36.387997 17749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:36.388610 17752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6934034d51fc4098bce1d56a7cf0e332" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34127 } }
I20260619 20:44:36.389225 17753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6934034d51fc4098bce1d56a7cf0e332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6934034d51fc4098bce1d56a7cf0e332" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34127 } } }
I20260619 20:44:36.389245 17754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6934034d51fc4098bce1d56a7cf0e332. Latest consensus state: current_term: 1 leader_uuid: "6934034d51fc4098bce1d56a7cf0e332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6934034d51fc4098bce1d56a7cf0e332" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 34127 } } }
I20260619 20:44:36.389313 17753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:36.389331 17754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:36.389534 17763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:36.390090 17763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:36.391783 17763 catalog_manager.cc:1347] Generated new cluster ID: eb8c18c6b513495b8f09469e97aa4e8c
I20260619 20:44:36.391844 17763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:36.397339 17763 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:36.398033 17763 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:36.400548 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.388910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.13.195.126:34127)
0619 20:44:36.389152 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:36.389162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:36.389542 (+ 380us) server_negotiation.cc:407] Connection header received
0619 20:44:36.389787 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:36.389796 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:36.389882 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:36.390007 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:36.390976 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.392324 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.393372 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.394042 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.394159 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.396967 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:36.396971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:36.396971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:36.396983 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:36.398954 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.399369 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.399372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.399372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.399472 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.399682 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.399683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.399683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.399933 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:36.400080 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:36.400202 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 20:44:36.400337 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
I20260619 20:44:36.402388 17763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6934034d51fc4098bce1d56a7cf0e332: Generated new TSK 0
I20260619 20:44:36.402571 17763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:36.429762 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34127
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:36.514807 17775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:36.515018 17775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:36.515045 17775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:36.516268 17775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:36.516314 17775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:36.516340 17775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:36.517911 17775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:36.518177 17775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:36.518487 17775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:36.523530 17780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.523530 17781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.523572 17783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:36.524082 17775 server_base.cc:1034] running on GCE node
I20260619 20:44:36.524304 17775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:36.524511 17775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:36.525681 17775 hybrid_clock.cc:648] HybridClock initialized: now 1781901876525667 us; error 35 us; skew 500 ppm
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:36.528589 17775 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:36.528998 17775 webserver.cc:460] Webserver started at http://127.13.195.65:45107/ using document root <none> and password file <none>
I20260619 20:44:36.529213 17775 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:36.529258 17775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:36.529362 17775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:36.530400 17775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "9753acee89c1404b8b610860ed9d62e3"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "adc1b824ce3d7de14e32028ae04e56f3"
server_key_iv: "3a9f7e6dfde11526d0dd7853cbab9d9a"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.530805 17775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "9753acee89c1404b8b610860ed9d62e3"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "adc1b824ce3d7de14e32028ae04e56f3"
server_key_iv: "3a9f7e6dfde11526d0dd7853cbab9d9a"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.532138 17775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:36.532884 17790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.533056 17775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:36.533170 17775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "9753acee89c1404b8b610860ed9d62e3"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "adc1b824ce3d7de14e32028ae04e56f3"
server_key_iv: "3a9f7e6dfde11526d0dd7853cbab9d9a"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.533247 17775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:36.566021 17775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:36.568598 17775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:36.568773 17775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:36.569023 17775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:36.569357 17775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:36.569412 17775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.569458 17775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:36.569491 17775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.575295 17775 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:45645
I20260619 20:44:36.575342 17904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:45645 every 8 connection(s)
I20260619 20:44:36.575675 17775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:36.584611 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.576415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:48741 (local address 127.13.195.126:34127)
0619 20:44:36.576464 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:36.576466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:36.576669 (+ 203us) server_negotiation.cc:407] Connection header received
0619 20:44:36.577563 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:36.577565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:36.577575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:36.577643 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:36.578256 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.578794 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.579815 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.580027 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.580086 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.582212 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:36.582214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:36.582214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:36.582220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:36.583488 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.583842 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.583843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.583843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.583894 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.584105 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.584106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.584106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.584204 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:36.584254 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:36.584476 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 20:44:36.584545 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:44:36.584770 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.576369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34127 (local address 127.13.195.65:48741)
0619 20:44:36.576528 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:36.576572 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:36.577472 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:36.577700 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:36.577704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:36.577949 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:36.578140 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:36.578144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.578897 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.578898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:36.579717 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:36.579719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.580107 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.580108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:36.580208 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.580788 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:36.580797 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:36.582124 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:36.583562 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:36.583567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:36.583569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:36.583773 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:36.583952 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:36.583953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:36.583953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:36.584051 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:36.584260 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:36.584264 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:36.584373 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 20:44:36.584459 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:44:36.584573 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260619 20:44:36.585007 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17775
I20260619 20:44:36.585106 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:36.585309 14093 external_mini_cluster.cc:1428] Setting key 87eb920ee41757cb641828a0ca647cd9
I20260619 20:44:36.585321 17905 heartbeater.cc:347] Connected to a master server at 127.13.195.126:34127
I20260619 20:44:36.585446 17905 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:36.585834 17905 heartbeater.cc:511] Master 127.13.195.126:34127 requested a full tablet report, sending...
I20260619 20:44:36.586468 17712 ts_manager.cc:194] Registered new tserver with Master: 9753acee89c1404b8b610860ed9d62e3 (127.13.195.65:45645)
I20260619 20:44:36.587381 17712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:48741
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:36.609755 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34127
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:36.692032 17911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:36.692268 17911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:36.692305 17911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:36.693499 17911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:36.693564 17911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:36.693611 17911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:36.695152 17911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:36.695431 17911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:36.695669 17911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:36.700635 17917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.700642 17919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.700642 17916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:36.700971 17911 server_base.cc:1034] running on GCE node
I20260619 20:44:36.701200 17911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:36.701436 17911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:36.702596 17911 hybrid_clock.cc:648] HybridClock initialized: now 1781901876702591 us; error 34 us; skew 500 ppm
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:36.705164 17911 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:36.705600 17911 webserver.cc:460] Webserver started at http://127.13.195.66:38241/ using document root <none> and password file <none>
I20260619 20:44:36.705794 17911 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:36.705833 17911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:36.705919 17911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:36.706972 17911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "cdc6f4b82ac64a6da627fb0c7bbf205b"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "2d85d0c0a4493d8adbeb854c156b6b66"
server_key_iv: "9a49f07dc5207517d0faf0eaa2b05e60"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.707280 17911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "cdc6f4b82ac64a6da627fb0c7bbf205b"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "2d85d0c0a4493d8adbeb854c156b6b66"
server_key_iv: "9a49f07dc5207517d0faf0eaa2b05e60"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.708513 17911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:36.709187 17926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.709394 17911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:36.709475 17911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "cdc6f4b82ac64a6da627fb0c7bbf205b"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "2d85d0c0a4493d8adbeb854c156b6b66"
server_key_iv: "9a49f07dc5207517d0faf0eaa2b05e60"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.709542 17911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:36.718494 17911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:36.720444 17911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:36.720558 17911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:36.720726 17911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:36.721021 17911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:36.721055 17911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.721108 17911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:36.721141 17911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.726558 17911 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:37663
I20260619 20:44:36.726637 18040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:37663 every 8 connection(s)
I20260619 20:44:36.726933 17911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:36.734562 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 17911
I20260619 20:44:36.734679 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:36.734876 14093 external_mini_cluster.cc:1428] Setting key 07affaea8e6317a0f1c1af663f41414c
I20260619 20:44:36.735970 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.727662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:35745 (local address 127.13.195.126:34127)
0619 20:44:36.727754 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:36.727756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:36.727898 (+ 142us) server_negotiation.cc:407] Connection header received
0619 20:44:36.728790 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:36.728791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:36.728802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:36.728862 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:36.729424 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.729899 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.730859 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.731047 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.731102 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.733211 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:36.733212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:36.733213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:36.733220 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:36.734504 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.734976 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.734978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.734978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.735045 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.735303 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.735304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.735304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.735420 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:36.735492 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:36.735777 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 20:44:36.735855 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:44:36.736228 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.727596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34127 (local address 127.13.195.66:35745)
0619 20:44:36.727758 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:36.727800 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:36.728702 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:36.728917 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:36.728920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:36.729165 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:36.729352 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:36.729355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.729983 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.729983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:36.730786 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:36.730788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.731118 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.731119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:36.731213 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.731747 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:36.731755 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:36.733116 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:36.734593 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:36.734600 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:36.734602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:36.734890 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:36.735113 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:36.735114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:36.735114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:36.735228 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:36.735507 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:36.735513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:36.735648 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 20:44:36.735756 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 20:44:36.735955 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260619 20:44:36.736675 18041 heartbeater.cc:347] Connected to a master server at 127.13.195.126:34127
I20260619 20:44:36.736780 18041 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:36.736960 18041 heartbeater.cc:511] Master 127.13.195.126:34127 requested a full tablet report, sending...
I20260619 20:44:36.737406 17712 ts_manager.cc:194] Registered new tserver with Master: cdc6f4b82ac64a6da627fb0c7bbf205b (127.13.195.66:37663)
I20260619 20:44:36.737779 17712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:35745
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:36.759518 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34127
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:36.840195 18046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:36.840404 18046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:36.840427 18046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:36.841573 18046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:36.841614 18046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:36.841639 18046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:36.843160 18046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:34127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:36.843405 18046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:36.843631 18046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:36.848556 18052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.848551 18054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:36.848551 18051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:36.848867 18046 server_base.cc:1034] running on GCE node
I20260619 20:44:36.849092 18046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:36.849323 18046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:36.850502 18046 hybrid_clock.cc:648] HybridClock initialized: now 1781901876850478 us; error 35 us; skew 500 ppm
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:36.853044 18046 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:36.853454 18046 webserver.cc:460] Webserver started at http://127.13.195.67:36043/ using document root <none> and password file <none>
I20260619 20:44:36.853689 18046 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:36.853768 18046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:36.853891 18046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:36.854925 18046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "78bcc3e7892e4219be95386bc7541d02"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "ff3506ef2e1b65b1bf13a2fc86be31b4"
server_key_iv: "09fe8e37265ae31379d800ee55ac705a"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.855227 18046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "78bcc3e7892e4219be95386bc7541d02"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "ff3506ef2e1b65b1bf13a2fc86be31b4"
server_key_iv: "09fe8e37265ae31379d800ee55ac705a"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.856500 18046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:36.857223 18061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.857409 18046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:36.857488 18046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "78bcc3e7892e4219be95386bc7541d02"
format_stamp: "Formatted at 2026-06-19 20:44:36 on dist-test-slave-n2td"
server_key: "ff3506ef2e1b65b1bf13a2fc86be31b4"
server_key_iv: "09fe8e37265ae31379d800ee55ac705a"
server_key_version: "encryptionkey@0"
I20260619 20:44:36.857542 18046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:36.878988 18046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:36.881287 18046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:36.881399 18046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:36.881578 18046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:36.881901 18046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:36.881942 18046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.882002 18046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:36.882032 18046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:36.887761 18046 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:33595
I20260619 20:44:36.887836 18175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:33595 every 8 connection(s)
I20260619 20:44:36.888180 18046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:36.894423 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18046
I20260619 20:44:36.894595 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:36.894825 14093 external_mini_cluster.cc:1428] Setting key d51f2cc504314f9b953988d6ac941b9e
I20260619 20:44:36.897374 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.888903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:55317 (local address 127.13.195.126:34127)
0619 20:44:36.888973 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:36.888975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:36.889240 (+ 265us) server_negotiation.cc:407] Connection header received
0619 20:44:36.890244 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:36.890245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:36.890256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:36.890309 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:36.890964 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.891505 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.892420 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.892601 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.892650 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.894745 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:36.894748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:36.894748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:36.894756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:36.896241 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.896597 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.896598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.896598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.896647 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.896845 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.896846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.896846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.896941 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:36.897035 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:36.897248 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:44:36.897310 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:44:36.897548 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.888905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:34127 (local address 127.13.195.67:55317)
0619 20:44:36.889093 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:36.889140 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:36.890153 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:36.890364 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:36.890368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:36.890685 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:36.890873 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:36.890876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.891581 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.891582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:36.892347 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:36.892349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.892669 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.892669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:36.892761 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.893188 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:36.893196 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:36.894600 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:36.896309 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:36.896314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:36.896315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:36.896533 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:36.896702 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:36.896703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:36.896703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:36.896794 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:36.897023 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:36.897027 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:36.897135 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 20:44:36.897214 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:44:36.897326 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260619 20:44:36.897995 18176 heartbeater.cc:347] Connected to a master server at 127.13.195.126:34127
I20260619 20:44:36.898075 18176 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:36.898267 18176 heartbeater.cc:511] Master 127.13.195.126:34127 requested a full tablet report, sending...
I20260619 20:44:36.898698 17712 ts_manager.cc:194] Registered new tserver with Master: 78bcc3e7892e4219be95386bc7541d02 (127.13.195.67:33595)
I20260619 20:44:36.899036 17712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:55317
I20260619 20:44:36.906570 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:36.929841 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.921432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59144 (local address 127.13.195.65:45645)
0619 20:44:36.921610 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:36.921617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:36.921633 (+ 16us) server_negotiation.cc:407] Connection header received
0619 20:44:36.921660 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:36.921662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:36.921676 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:36.921779 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:36.922422 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.923246 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.924108 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.924471 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.924573 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.927049 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:36.927057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:36.927057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:36.927064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:36.928821 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.929131 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.929134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.929135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.929192 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.929344 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.929345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.929345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.929515 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:36.929632 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:36.929690 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 20:44:36.929754 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
Jun 19 20:44:36 dist-test-slave-n2td krb5kdc[17667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:36.938050 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:36.931218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48928 (local address 127.13.195.126:34127)
0619 20:44:36.931280 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:36.931282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:36.931293 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:44:36.931324 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:36.931324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:36.931333 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:36.931376 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:36.931816 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.932284 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.932939 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:36.933119 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:36.933169 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:36.935481 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:36.935482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:36.935482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:36.935489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:36.936881 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.937246 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.937247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.937247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.937306 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:36.937508 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:36.937509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:36.937509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:36.937661 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:36.937761 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:36.937867 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 20:44:36.937948 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 20:44:36.938270 17712 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48928
I20260619 20:44:36.938632 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17775
I20260619 20:44:36.943648 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17911
I20260619 20:44:36.947901 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18046
I20260619 20:44:36.952210 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 17680
2026-06-19T20:44:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1860 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:37 dist-test-slave-n2td krb5kdc[18188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:37 dist-test-slave-n2td krb5kdc[18188](info): set up 2 sockets
Jun 19 20:44:37 dist-test-slave-n2td krb5kdc[18188](info): commencing operation
krb5kdc: starting...
W20260619 20:44:38.066051 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:38.179414 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:46367
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:46367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:38.260295 18201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:38.260560 18201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:38.260599 18201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:38.261807 18201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:38.261883 18201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:38.261914 18201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:38.261947 18201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:38.261971 18201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:38.263573 18201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:46367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:46367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:38.263862 18201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:38.264106 18201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:38.268874 18209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.268879 18207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.268930 18206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:38.269304 18201 server_base.cc:1034] running on GCE node
I20260619 20:44:38.269589 18201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:38.269865 18201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:38.271044 18201 hybrid_clock.cc:648] HybridClock initialized: now 1781901878271030 us; error 43 us; skew 500 ppm
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:38.277022 18201 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:38.277418 18201 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:38.277493 18201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:38.277648 18201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:38.278808 18201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "062c4a9b68f04e79a6c17ef3afa5bade"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "52cfff38a44412b8ee15c6383c64de41"
server_key_iv: "a8e616db34525bc52f26a6915b34a546"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.279186 18201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "062c4a9b68f04e79a6c17ef3afa5bade"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "52cfff38a44412b8ee15c6383c64de41"
server_key_iv: "a8e616db34525bc52f26a6915b34a546"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.280493 18201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:38.281213 18215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.281396 18201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:38.281497 18201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "062c4a9b68f04e79a6c17ef3afa5bade"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "52cfff38a44412b8ee15c6383c64de41"
server_key_iv: "a8e616db34525bc52f26a6915b34a546"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.281579 18201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:38.294981 18201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:38.296799 18201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:38.296963 18201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:38.301146 18201 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:46367
I20260619 20:44:38.301170 18267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:46367 every 8 connection(s)
I20260619 20:44:38.301453 18201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:38.302359 18268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:38.304075 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18201
I20260619 20:44:38.304157 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:38.304339 14093 external_mini_cluster.cc:1428] Setting key 78e5d5128e6e3892c43fec12164ef46b
I20260619 20:44:38.304945 18268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade: Bootstrap starting.
I20260619 20:44:38.305691 18268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:38.306041 18268 log.cc:824] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:38.306843 18268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade: No bootstrap required, opened a new log
I20260619 20:44:38.309062 18268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46367 } }
I20260619 20:44:38.309230 18268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:38.309263 18268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 062c4a9b68f04e79a6c17ef3afa5bade, State: Initialized, Role: FOLLOWER
I20260619 20:44:38.309394 18268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46367 } }
I20260619 20:44:38.309522 18268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:38.309598 18268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:38.309652 18268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:38.310173 18268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46367 } }
I20260619 20:44:38.310293 18268 leader_election.cc:302] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 062c4a9b68f04e79a6c17ef3afa5bade; no voters:
I20260619 20:44:38.310489 18268 leader_election.cc:288] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:38.310496 18274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:38.310628 18274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [term 1 LEADER]: Becoming Leader. State: Replica: 062c4a9b68f04e79a6c17ef3afa5bade, State: Running, Role: LEADER
I20260619 20:44:38.310768 18268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:38.310768 18274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46367 } }
I20260619 20:44:38.311283 18276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46367 } } }
I20260619 20:44:38.311445 18276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [sys.catalog]: This master's current role is: LEADER
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:38.311890 18283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:38.312121 18275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 062c4a9b68f04e79a6c17ef3afa5bade. Latest consensus state: current_term: 1 leader_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062c4a9b68f04e79a6c17ef3afa5bade" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46367 } } }
I20260619 20:44:38.312273 18275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:38.312460 18283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:38.314069 18283 catalog_manager.cc:1347] Generated new cluster ID: e8dee1e163684e5d891f0670b69cb507
I20260619 20:44:38.314121 18283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:38.315948 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.304861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.13.195.126:46367)
0619 20:44:38.305050 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.305060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.305093 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:44:38.305701 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.305710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.305780 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.305970 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:38.306582 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.307928 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.308683 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.309328 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.309434 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.312333 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:38.312337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:38.312337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:38.312350 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:38.314387 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.314782 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.314785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.314786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.314881 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.315091 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.315092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.315092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.315331 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:38.315466 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.315591 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.315726 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260619 20:44:38.322774 18283 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:38.323463 18283 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:38.327900 18283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade: Generated new TSK 0
I20260619 20:44:38.328094 18283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:38.363211 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46367
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:38.468310 18295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:38.468501 18295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:38.468524 18295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:38.469775 18295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:38.469828 18295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:38.469856 18295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:38.471356 18295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:38.471609 18295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:38.471858 18295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:38.477448 18301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.477483 18300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.477460 18303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:38.478087 18295 server_base.cc:1034] running on GCE node
I20260619 20:44:38.478319 18295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:38.478610 18295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:38.479748 18295 hybrid_clock.cc:648] HybridClock initialized: now 1781901878479734 us; error 29 us; skew 500 ppm
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:38.482213 18295 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:38.482573 18295 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:38.482621 18295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:38.482712 18295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:38.483789 18295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "837b3db8cd684232b3dbc1fd5b59f6b9"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "1f74acbe41e90d7be49c055695ba9ca2"
server_key_iv: "74dcf0285af20248571dd1442c0d14d4"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.484110 18295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "837b3db8cd684232b3dbc1fd5b59f6b9"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "1f74acbe41e90d7be49c055695ba9ca2"
server_key_iv: "74dcf0285af20248571dd1442c0d14d4"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.485520 18295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:44:38.486173 18309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.486356 18295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:38.486478 18295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "837b3db8cd684232b3dbc1fd5b59f6b9"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "1f74acbe41e90d7be49c055695ba9ca2"
server_key_iv: "74dcf0285af20248571dd1442c0d14d4"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.486558 18295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:38.509449 18295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:38.511611 18295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:38.511778 18295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:38.512063 18295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:38.512403 18295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:38.512459 18295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.512501 18295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:38.512527 18295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.518694 18295 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:37533
I20260619 20:44:38.518787 18422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:37533 every 8 connection(s)
I20260619 20:44:38.519187 18295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:38.528515 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.520197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:41305 (local address 127.13.195.126:46367)
0619 20:44:38.520258 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.520260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.520406 (+ 146us) server_negotiation.cc:407] Connection header received
0619 20:44:38.521004 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.521005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.521019 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.521106 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:38.522259 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.522835 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.523813 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.524014 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.524065 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.526119 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:38.526121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:38.526121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:38.526128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:38.527389 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.527735 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.527736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.527736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.527784 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.527983 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.527984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.527984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.528098 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:38.528147 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.528374 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.528435 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:44:38.528674 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.519953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:46367 (local address 127.13.195.65:41305)
0619 20:44:38.520234 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.520284 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.520883 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.521207 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.521211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.521587 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:38.522165 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.522172 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.522919 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.522920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.523736 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.523738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.524086 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.524086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.524184 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.524635 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:38.524643 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:38.526036 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:38.527458 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:38.527464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:38.527465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:38.527674 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:38.527840 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:38.527840 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:38.527840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:38.527933 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:38.528154 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:38.528158 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:38.528267 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 20:44:38.528348 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:44:38.528465 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260619 20:44:38.529196 18423 heartbeater.cc:347] Connected to a master server at 127.13.195.126:46367
I20260619 20:44:38.529279 18423 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:38.529307 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18295
I20260619 20:44:38.529419 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:38.529464 18423 heartbeater.cc:511] Master 127.13.195.126:46367 requested a full tablet report, sending...
I20260619 20:44:38.529615 14093 external_mini_cluster.cc:1428] Setting key 355e86946bc32751ceb62f7cbf90b688
I20260619 20:44:38.530014 18232 ts_manager.cc:194] Registered new tserver with Master: 837b3db8cd684232b3dbc1fd5b59f6b9 (127.13.195.65:37533)
I20260619 20:44:38.530813 18232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:41305
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:38.552878 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46367
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:38.632617 18428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:38.632846 18428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:38.632884 18428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:38.634016 18428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:38.634083 18428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:38.634130 18428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:38.635656 18428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:38.635915 18428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:38.636149 18428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:38.640961 18436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.640977 18434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.641016 18433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:38.641155 18428 server_base.cc:1034] running on GCE node
I20260619 20:44:38.641502 18428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:38.641754 18428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:38.642910 18428 hybrid_clock.cc:648] HybridClock initialized: now 1781901878642903 us; error 29 us; skew 500 ppm
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:38.645257 18428 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:38.645609 18428 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:38.645658 18428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:38.645745 18428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:38.646770 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "7af90ff1db844a468a282d3c4e147476"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "9261a3436c26e22cb71ad3f6c1f9b800"
server_key_iv: "3bca26bcf739f369a6a91bce8f8c5886"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.647128 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "7af90ff1db844a468a282d3c4e147476"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "9261a3436c26e22cb71ad3f6c1f9b800"
server_key_iv: "3bca26bcf739f369a6a91bce8f8c5886"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.648363 18428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:38.649051 18442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.649389 18428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:38.649485 18428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "7af90ff1db844a468a282d3c4e147476"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "9261a3436c26e22cb71ad3f6c1f9b800"
server_key_iv: "3bca26bcf739f369a6a91bce8f8c5886"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.649539 18428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:38.658696 18428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:38.660535 18428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:38.660640 18428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:38.660802 18428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:38.661118 18428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:38.661152 18428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.661173 18428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:38.661221 18428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.666955 18428 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:32915
I20260619 20:44:38.666988 18557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:32915 every 8 connection(s)
I20260619 20:44:38.667255 18428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:38.667492 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18428
I20260619 20:44:38.667641 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:38.667845 14093 external_mini_cluster.cc:1428] Setting key b84b8969460cc8069d30f9dcebd3922a
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:44:38.678553 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.668156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:57143 (local address 127.13.195.126:46367)
0619 20:44:38.668200 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.668202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.668428 (+ 226us) server_negotiation.cc:407] Connection header received
0619 20:44:38.669126 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.669127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.669137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.669191 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:38.670489 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.671272 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.672906 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.673151 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.673230 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.675702 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:38.675704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:38.675704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:38.675713 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:38.677163 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.677563 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.677564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.677565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.677623 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.677871 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.677872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.677872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.677987 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:38.678077 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.678344 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.678423 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:44:38.678742 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.668154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:46367 (local address 127.13.195.66:57143)
0619 20:44:38.668303 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.668349 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.668996 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.669272 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.669276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.669665 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:38.670346 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.670352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.671384 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.671385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.672814 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.672818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.673253 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.673254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.673432 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.673942 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:38.673953 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:38.675602 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:38.677255 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:38.677262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:38.677264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:38.677480 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:38.677690 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:38.677691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:38.677691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:38.677802 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:38.678067 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:38.678073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:38.678215 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 20:44:38.678324 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:44:38.678505 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260619 20:44:38.679301 18558 heartbeater.cc:347] Connected to a master server at 127.13.195.126:46367
I20260619 20:44:38.679382 18558 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:38.679538 18558 heartbeater.cc:511] Master 127.13.195.126:46367 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
I20260619 20:44:38.679947 18232 ts_manager.cc:194] Registered new tserver with Master: 7af90ff1db844a468a282d3c4e147476 (127.13.195.66:32915)
I20260619 20:44:38.680339 18232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:57143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:38.693267 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46367
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:38.773265 18564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:38.773473 18564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:38.773499 18564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:38.774703 18564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:38.774746 18564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:38.774775 18564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:38.776281 18564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:38.776530 18564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:38.776767 18564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:38.781644 18572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.781692 18570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:38.781723 18569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:38.782016 18564 server_base.cc:1034] running on GCE node
I20260619 20:44:38.782265 18564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:38.782569 18564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:38.783731 18564 hybrid_clock.cc:648] HybridClock initialized: now 1781901878783725 us; error 36 us; skew 500 ppm
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:38.786221 18564 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:38.786607 18564 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:38.786680 18564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:38.786832 18564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:38.787935 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "83c9ac23e4594958bc423c87d29223c8"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "695d73f4f73fe01af2291b9323c9d1f7"
server_key_iv: "8fd95f7d1da057b5160d4cd95a79794c"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.788306 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "83c9ac23e4594958bc423c87d29223c8"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "695d73f4f73fe01af2291b9323c9d1f7"
server_key_iv: "8fd95f7d1da057b5160d4cd95a79794c"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.789655 18564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:38.790409 18578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.790606 18564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:38.790701 18564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "83c9ac23e4594958bc423c87d29223c8"
format_stamp: "Formatted at 2026-06-19 20:44:38 on dist-test-slave-n2td"
server_key: "695d73f4f73fe01af2291b9323c9d1f7"
server_key_iv: "8fd95f7d1da057b5160d4cd95a79794c"
server_key_version: "encryptionkey@0"
I20260619 20:44:38.790786 18564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:38.810911 18564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:38.813318 18564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:38.813485 18564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:38.813719 18564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:38.814052 18564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:38.814109 18564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.814152 18564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:38.814178 18564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:38.820144 18564 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:40913
I20260619 20:44:38.820180 18691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:40913 every 8 connection(s)
I20260619 20:44:38.820506 18564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:44:38 dist-test-slave-n2td krb5kdc[18188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:38.828322 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18564
I20260619 20:44:38.828470 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:38.828687 14093 external_mini_cluster.cc:1428] Setting key 437759dedd15ca30d80331b909e3fbdd
I20260619 20:44:38.830116 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.821234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:39773 (local address 127.13.195.126:46367)
0619 20:44:38.821295 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.821298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.821531 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:44:38.822005 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.822006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.822015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.822059 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:38.823033 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.823530 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.824545 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.824721 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.824772 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.826814 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:38.826815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:38.826816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:38.826830 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:38.828259 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.828861 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.828862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.828863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.828937 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.829364 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.829365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.829365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.829530 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:38.829617 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.829940 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.830023 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:44:38.830330 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.821224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:46367 (local address 127.13.195.67:39773)
0619 20:44:38.821396 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.821438 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.821904 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.822113 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.822117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.822364 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:38.822922 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.822927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.823612 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.823613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.824471 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.824473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.824792 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.824793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.824890 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.825338 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:38.825346 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:38.826710 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:38.828410 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:38.828418 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:38.828420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:38.828761 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:38.829046 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:38.829047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:38.829048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:38.829220 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:38.829608 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:38.829616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:38.829787 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 20:44:38.829906 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:44:38.830075 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260619 20:44:38.830878 18692 heartbeater.cc:347] Connected to a master server at 127.13.195.126:46367
I20260619 20:44:38.830983 18692 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:38.831172 18692 heartbeater.cc:511] Master 127.13.195.126:46367 requested a full tablet report, sending...
I20260619 20:44:38.831591 18232 ts_manager.cc:194] Registered new tserver with Master: 83c9ac23e4594958bc423c87d29223c8 (127.13.195.67:40913)
I20260619 20:44:38.831974 18232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:39773
I20260619 20:44:38.840791 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:44:38.847676 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.842073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50846 (local address 127.13.195.126:46367)
0619 20:44:38.842134 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.842136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.842170 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:44:38.842243 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.842243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.842253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.842301 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:38.842944 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.843423 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.844107 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.844280 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.844329 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.845435 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:38.845436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:38.845436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:38.845442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:38.846681 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.846974 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.846975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.846975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.847022 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:38.847179 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:38.847179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:38.847179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:38.847314 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:38.847387 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.847507 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.847570 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:44:38.849342 18232 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:38.851012 18232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:38.857611 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.853306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:32915 (local address 127.0.0.1:42382)
0619 20:44:38.853592 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.853603 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.853652 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.854012 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.854015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.854017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:38.854087 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.854089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.855473 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.855474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.856439 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.856442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.857304 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.857304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.857391 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.857407 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 20:44:38.857479 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:44:38.857528 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":95,"threads_started":1}
I20260619 20:44:38.857709 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.853407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.13.195.66:32915)
0619 20:44:38.853833 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.853837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.853846 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:38.853889 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.853891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.853903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.853968 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:38.854175 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.855336 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.856535 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.857220 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.857297 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.857352 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.857495 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.857566 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":99,"threads_started":1}
I20260619 20:44:38.858843 18492 tablet_service.cc:1467] Processing CreateTablet for tablet b2f333cb77ec4b029ab46d1f7f6e3445 (DEFAULT_TABLE table=test-table [id=52779469f3a1405fa1f213d886e7a542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:38.859010 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.853494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40913 (local address 127.0.0.1:50478)
0619 20:44:38.854106 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.854116 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.854164 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.854356 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.854357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.854358 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:38.854403 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.854404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.855266 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.855267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.856670 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.856671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.858745 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.858746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.858825 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.858829 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:44:38.858885 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:44:38.858929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":85,"thread_start_us":64,"threads_started":1}
I20260619 20:44:38.859256 18492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f333cb77ec4b029ab46d1f7f6e3445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:38.859403 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.853619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50478 (local address 127.13.195.67:40913)
0619 20:44:38.853785 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.853791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.854225 (+ 434us) server_negotiation.cc:407] Connection header received
0619 20:44:38.854270 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.854273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.854288 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.854359 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:38.854500 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.855166 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.857593 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.858654 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.858712 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.858764 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.858921 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.858960 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260619 20:44:38.860260 18626 tablet_service.cc:1467] Processing CreateTablet for tablet b2f333cb77ec4b029ab46d1f7f6e3445 (DEFAULT_TABLE table=test-table [id=52779469f3a1405fa1f213d886e7a542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:38.860594 18626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f333cb77ec4b029ab46d1f7f6e3445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:38.863246 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.853285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:37533 (local address 127.0.0.1:36894)
0619 20:44:38.853450 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.853494 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.853562 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.854078 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.854079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.854080 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:38.854121 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.854123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.855437 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.855438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.856669 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.856672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.862858 (+ 6186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.862860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.863048 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.863053 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:44:38.863121 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:44:38.863169 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260619 20:44:38.863306 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.853704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.13.195.65:37533)
0619 20:44:38.853865 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.853870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.853879 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:38.853919 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.853921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.853932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.854196 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:38.854278 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.855346 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.856753 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.857434 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.857508 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:38.857559 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:38.863152 (+ 5593us) server_negotiation.cc:299] Negotiation successful
0619 20:44:38.863209 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260619 20:44:38.863837 18710 tablet_bootstrap.cc:492] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Bootstrap starting.
I20260619 20:44:38.863906 18357 tablet_service.cc:1467] Processing CreateTablet for tablet b2f333cb77ec4b029ab46d1f7f6e3445 (DEFAULT_TABLE table=test-table [id=52779469f3a1405fa1f213d886e7a542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:38.864233 18357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f333cb77ec4b029ab46d1f7f6e3445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:38.864398 18710 tablet_bootstrap.cc:654] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:38.864831 18710 log.cc:824] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:38.865545 18709 tablet_bootstrap.cc:492] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Bootstrap starting.
I20260619 20:44:38.865607 18710 tablet_bootstrap.cc:492] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: No bootstrap required, opened a new log
I20260619 20:44:38.865675 18710 ts_tablet_manager.cc:1397] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:38.866315 18709 tablet_bootstrap.cc:654] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:38.867017 18709 log.cc:824] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:38.867545 18710 raft_consensus.cc:348] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:38.867722 18710 raft_consensus.cc:374] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:38.867707 18713 tablet_bootstrap.cc:492] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: Bootstrap starting.
I20260619 20:44:38.867764 18710 raft_consensus.cc:729] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c9ac23e4594958bc423c87d29223c8, State: Initialized, Role: FOLLOWER
I20260619 20:44:38.867873 18710 consensus_queue.cc:260] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:38.868021 18709 tablet_bootstrap.cc:492] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: No bootstrap required, opened a new log
I20260619 20:44:38.868104 18709 ts_tablet_manager.cc:1397] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:44:38.868183 18710 ts_tablet_manager.cc:1428] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:44:38.868330 18692 heartbeater.cc:503] Master 127.13.195.126:46367 was elected leader, sending a full tablet report...
I20260619 20:44:38.868357 18713 tablet_bootstrap.cc:654] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:38.868800 18713 log.cc:824] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:38.869469 18713 tablet_bootstrap.cc:492] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: No bootstrap required, opened a new log
I20260619 20:44:38.869541 18713 ts_tablet_manager.cc:1397] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:38.870167 18709 raft_consensus.cc:348] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:38.870332 18709 raft_consensus.cc:374] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:38.870368 18709 raft_consensus.cc:729] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af90ff1db844a468a282d3c4e147476, State: Initialized, Role: FOLLOWER
I20260619 20:44:38.870532 18709 consensus_queue.cc:260] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:38.870798 18709 ts_tablet_manager.cc:1428] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:44:38.870914 18558 heartbeater.cc:503] Master 127.13.195.126:46367 was elected leader, sending a full tablet report...
I20260619 20:44:38.871762 18713 raft_consensus.cc:348] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:38.871924 18713 raft_consensus.cc:374] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:38.871973 18713 raft_consensus.cc:729] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 837b3db8cd684232b3dbc1fd5b59f6b9, State: Initialized, Role: FOLLOWER
I20260619 20:44:38.872092 18713 consensus_queue.cc:260] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:38.872361 18713 ts_tablet_manager.cc:1428] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:44:38.872388 18423 heartbeater.cc:503] Master 127.13.195.126:46367 was elected leader, sending a full tablet report...
W20260619 20:44:38.917886 18559 tablet.cc:2369] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:38.996840 18714 raft_consensus.cc:482] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:38.996960 18714 raft_consensus.cc:504] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:38.997457 18714 leader_election.cc:288] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 837b3db8cd684232b3dbc1fd5b59f6b9 (127.13.195.65:37533), 7af90ff1db844a468a282d3c4e147476 (127.13.195.66:32915)
I20260619 20:44:39.000442 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.997615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:37533 (local address 127.13.195.67:39149)
0619 20:44:38.997680 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.997691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.997728 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.997876 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.997877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.997878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:38.997965 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.997966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.998696 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.998697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.999614 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.999617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.000170 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.000171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:39.000260 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.000263 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.000330 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.000366 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":38}
I20260619 20:44:39.000442 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.997669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:39149 (local address 127.13.195.65:37533)
0619 20:44:38.997707 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.997709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.997734 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:44:38.997790 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.997791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.997800 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.997872 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:38.998040 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.998611 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.999708 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.000072 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.000171 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.000180 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.000335 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.000371 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:39.000561 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.997710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:42247 (local address 127.13.195.66:32915)
0619 20:44:38.997757 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:38.997760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:38.997902 (+ 142us) server_negotiation.cc:407] Connection header received
0619 20:44:38.997978 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:38.997979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:38.997993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:38.998052 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:38.998202 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.998830 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.999809 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.000195 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.000279 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.000287 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.000452 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.000500 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:44:39.000551 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:38.997660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:32915 (local address 127.13.195.67:42247)
0619 20:44:38.997814 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:38.997823 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:38.997911 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:38.998053 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:38.998054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:38.998056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:38.998131 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.998132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:38.998937 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:38.998938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:38.999716 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:38.999718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.000291 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.000292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:39.000371 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.000374 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.000432 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.000473 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260619 20:44:39.000805 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f333cb77ec4b029ab46d1f7f6e3445" candidate_uuid: "83c9ac23e4594958bc423c87d29223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" is_pre_election: true
I20260619 20:44:39.000963 18377 raft_consensus.cc:2454] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83c9ac23e4594958bc423c87d29223c8 in term 0.
I20260619 20:44:39.001216 18581 leader_election.cc:302] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 837b3db8cd684232b3dbc1fd5b59f6b9, 83c9ac23e4594958bc423c87d29223c8; no voters:
I20260619 20:44:39.001237 18512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f333cb77ec4b029ab46d1f7f6e3445" candidate_uuid: "83c9ac23e4594958bc423c87d29223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af90ff1db844a468a282d3c4e147476" is_pre_election: true
I20260619 20:44:39.001400 18512 raft_consensus.cc:2454] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83c9ac23e4594958bc423c87d29223c8 in term 0.
I20260619 20:44:39.001401 18714 raft_consensus.cc:2781] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:39.001482 18714 raft_consensus.cc:482] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:39.001511 18714 raft_consensus.cc:3037] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.002540 18714 raft_consensus.cc:504] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:39.002688 18714 leader_election.cc:288] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [CANDIDATE]: Term 1 election: Requested vote from peers 837b3db8cd684232b3dbc1fd5b59f6b9 (127.13.195.65:37533), 7af90ff1db844a468a282d3c4e147476 (127.13.195.66:32915)
I20260619 20:44:39.002846 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f333cb77ec4b029ab46d1f7f6e3445" candidate_uuid: "83c9ac23e4594958bc423c87d29223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9"
I20260619 20:44:39.002851 18512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f333cb77ec4b029ab46d1f7f6e3445" candidate_uuid: "83c9ac23e4594958bc423c87d29223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af90ff1db844a468a282d3c4e147476"
I20260619 20:44:39.002936 18512 raft_consensus.cc:3037] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.002955 18377 raft_consensus.cc:3037] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.003669 18512 raft_consensus.cc:2454] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c9ac23e4594958bc423c87d29223c8 in term 1.
I20260619 20:44:39.003674 18377 raft_consensus.cc:2454] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c9ac23e4594958bc423c87d29223c8 in term 1.
I20260619 20:44:39.003850 18580 leader_election.cc:302] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7af90ff1db844a468a282d3c4e147476, 83c9ac23e4594958bc423c87d29223c8; no voters:
I20260619 20:44:39.003985 18714 raft_consensus.cc:2781] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:39.004161 18714 raft_consensus.cc:686] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 1 LEADER]: Becoming Leader. State: Replica: 83c9ac23e4594958bc423c87d29223c8, State: Running, Role: LEADER
I20260619 20:44:39.004282 18714 consensus_queue.cc:237] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } }
I20260619 20:44:39.005214 18232 catalog_manager.cc:5495] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83c9ac23e4594958bc423c87d29223c8 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "83c9ac23e4594958bc423c87d29223c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83c9ac23e4594958bc423c87d29223c8" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40913 } health_report { overall_health: HEALTHY } } }
W20260619 20:44:39.020291 18424 tablet.cc:2369] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:39.039459 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.036964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.13.195.67:40913)
0619 20:44:39.037032 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.037034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.037045 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:44:39.037096 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.037097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.037115 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.037177 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:39.037335 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.037887 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.038647 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.038824 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.038877 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.039049 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:39.039155 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:39.039240 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.039258 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.039278 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:44:39.041301 18512 raft_consensus.cc:1261] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 1 FOLLOWER]: Refusing update from remote peer 83c9ac23e4594958bc423c87d29223c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:39.041308 18377 raft_consensus.cc:1261] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 1 FOLLOWER]: Refusing update from remote peer 83c9ac23e4594958bc423c87d29223c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:39.041658 18714 consensus_queue.cc:1035] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "837b3db8cd684232b3dbc1fd5b59f6b9" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:39.041806 18719 consensus_queue.cc:1035] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af90ff1db844a468a282d3c4e147476" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 32915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:39.044227 18724 mvcc.cc:205] Tried to move back new op lower bound from 7298670096550494208 to 7298670096402337792. Current Snapshot: MvccSnapshot[applied={T|T < 7298670096550494208}]
I20260619 20:44:39.046324 18232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:39.046439 18232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:39.047214 18232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 062c4a9b68f04e79a6c17ef3afa5bade: Sending DeleteTablet for 3 replicas of tablet b2f333cb77ec4b029ab46d1f7f6e3445
I20260619 20:44:39.047513 18357 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f333cb77ec4b029ab46d1f7f6e3445 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:39 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:36894
I20260619 20:44:39.047555 18492 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f333cb77ec4b029ab46d1f7f6e3445 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:39 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:42382
I20260619 20:44:39.047741 18732 tablet_replica.cc:330] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: stopping tablet replica
I20260619 20:44:39.047762 18626 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f333cb77ec4b029ab46d1f7f6e3445 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:39 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:50478
I20260619 20:44:39.047847 18732 raft_consensus.cc:2229] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:39.047959 18732 raft_consensus.cc:2258] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:39.048038 18733 tablet_replica.cc:330] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: stopping tablet replica
I20260619 20:44:39.048134 18733 raft_consensus.cc:2229] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:39.048225 18733 raft_consensus.cc:2258] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:39.048262 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18295
I20260619 20:44:39.048424 18732 ts_tablet_manager.cc:1905] T b2f333cb77ec4b029ab46d1f7f6e3445 P 837b3db8cd684232b3dbc1fd5b59f6b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:39.048804 18734 tablet_replica.cc:330] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: stopping tablet replica
I20260619 20:44:39.048893 18734 raft_consensus.cc:2229] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:39.048980 18734 raft_consensus.cc:2258] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:39.049428 18733 ts_tablet_manager.cc:1905] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:39.049428 18734 ts_tablet_manager.cc:1905] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:39.051146 18734 ts_tablet_manager.cc:1918] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:39.051156 18733 ts_tablet_manager.cc:1918] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:39.051213 18733 log.cc:1196] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/b2f333cb77ec4b029ab46d1f7f6e3445
I20260619 20:44:39.051213 18734 log.cc:1196] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/b2f333cb77ec4b029ab46d1f7f6e3445
I20260619 20:44:39.051487 18733 ts_tablet_manager.cc:1939] T b2f333cb77ec4b029ab46d1f7f6e3445 P 7af90ff1db844a468a282d3c4e147476: Deleting consensus metadata
I20260619 20:44:39.051487 18734 ts_tablet_manager.cc:1939] T b2f333cb77ec4b029ab46d1f7f6e3445 P 83c9ac23e4594958bc423c87d29223c8: Deleting consensus metadata
I20260619 20:44:39.051874 18218 catalog_manager.cc:4841] TS 83c9ac23e4594958bc423c87d29223c8 (127.13.195.67:40913): tablet b2f333cb77ec4b029ab46d1f7f6e3445 (table test-table [id=52779469f3a1405fa1f213d886e7a542]) successfully deleted
I20260619 20:44:39.051874 18217 catalog_manager.cc:4841] TS 7af90ff1db844a468a282d3c4e147476 (127.13.195.66:32915): tablet b2f333cb77ec4b029ab46d1f7f6e3445 (table test-table [id=52779469f3a1405fa1f213d886e7a542]) successfully deleted
W20260619 20:44:39.055979 18218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:37533 (error 108)
I20260619 20:44:39.056319 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18428
I20260619 20:44:39.056490 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.056355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:37533 (local address 127.0.0.1:36908)
0619 20:44:39.056417 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.056452 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 20:44:39.056927 18218 catalog_manager.cc:4584] TS 837b3db8cd684232b3dbc1fd5b59f6b9 (127.13.195.65:37533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2f333cb77ec4b029ab46d1f7f6e3445: Network error: Client connection negotiation failed: client connection to 127.13.195.65:37533: connect: Connection refused (error 111)
I20260619 20:44:39.063870 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18564
I20260619 20:44:39.068228 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18201
2026-06-19T20:44:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2115 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T20:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:39Z Disabled control of system clock
I20260619 20:44:39.094485 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:46805
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:37353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:46805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:44:39.174327 18740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:39.174598 18740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:39.174624 18740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:39.175753 18740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:39.175791 18740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:39.175804 18740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:39.175818 18740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:39.175830 18740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:39.177095 18740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:46805
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:46805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:39.177282 18740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:39.177508 18740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:39.182156 18748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.182164 18745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.182165 18746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.182458 18740 server_base.cc:1034] running on GCE node
I20260619 20:44:39.182729 18740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:39.182962 18740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:39.184101 18740 hybrid_clock.cc:648] HybridClock initialized: now 1781901879184083 us; error 46 us; skew 500 ppm
I20260619 20:44:39.185366 18740 webserver.cc:460] Webserver started at http://127.13.195.126:36699/ using document root <none> and password file <none>
I20260619 20:44:39.185616 18740 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:39.185684 18740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:39.185840 18740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:39.186978 18740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "ec8ed09abf414576a1e62386426d4ff5"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "7909b975928e87a9cdb3d065782c74d5"
server_key_iv: "835d909c819f7a4d5d637d93d9cea537"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.187338 18740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "ec8ed09abf414576a1e62386426d4ff5"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "7909b975928e87a9cdb3d065782c74d5"
server_key_iv: "835d909c819f7a4d5d637d93d9cea537"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.188710 18740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:39.189435 18755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.189596 18740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:39.189697 18740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "ec8ed09abf414576a1e62386426d4ff5"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "7909b975928e87a9cdb3d065782c74d5"
server_key_iv: "835d909c819f7a4d5d637d93d9cea537"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.189778 18740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:39.193090 18740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:39.193342 18740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:39.193495 18740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:39.197341 18740 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:46805
I20260619 20:44:39.197372 18807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:46805 every 8 connection(s)
I20260619 20:44:39.197700 18740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:39.198309 18808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:39.198433 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18740
I20260619 20:44:39.198544 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:39.198743 14093 external_mini_cluster.cc:1428] Setting key 5323935fb8a4ad83e799fa4f52065eff
I20260619 20:44:39.201414 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.199321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.13.195.126:46805)
0619 20:44:39.199547 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.199557 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.199592 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:44:39.200154 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.200163 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.200216 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.200366 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.201010 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.201015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.201015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.201019 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.201064 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.201077 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.201154 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.201190 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.201242 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260619 20:44:39.202414 18808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5: Bootstrap starting.
I20260619 20:44:39.203076 18808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:39.203354 18808 log.cc:824] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:39.204006 18808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5: No bootstrap required, opened a new log
I20260619 20:44:39.205328 18808 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ed09abf414576a1e62386426d4ff5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46805 } }
I20260619 20:44:39.205466 18808 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:39.205488 18808 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8ed09abf414576a1e62386426d4ff5, State: Initialized, Role: FOLLOWER
I20260619 20:44:39.205622 18808 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ed09abf414576a1e62386426d4ff5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46805 } }
I20260619 20:44:39.205703 18808 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:39.205746 18808 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:39.205796 18808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.206301 18808 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ed09abf414576a1e62386426d4ff5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46805 } }
I20260619 20:44:39.206415 18808 leader_election.cc:302] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec8ed09abf414576a1e62386426d4ff5; no voters:
I20260619 20:44:39.206605 18808 leader_election.cc:288] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:39.206878 18813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:39.206911 18808 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:39.206992 18813 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [term 1 LEADER]: Becoming Leader. State: Replica: ec8ed09abf414576a1e62386426d4ff5, State: Running, Role: LEADER
I20260619 20:44:39.207077 18813 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ed09abf414576a1e62386426d4ff5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46805 } }
I20260619 20:44:39.207475 18814 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec8ed09abf414576a1e62386426d4ff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ed09abf414576a1e62386426d4ff5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46805 } } }
I20260619 20:44:39.207475 18815 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec8ed09abf414576a1e62386426d4ff5. Latest consensus state: current_term: 1 leader_uuid: "ec8ed09abf414576a1e62386426d4ff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ed09abf414576a1e62386426d4ff5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46805 } } }
I20260619 20:44:39.207602 18814 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:39.207613 18815 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:39.207798 18822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:39.208305 18822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:39.209700 18822 catalog_manager.cc:1347] Generated new cluster ID: b7237407f89840d1bed819929a64a251
I20260619 20:44:39.209753 18822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:39.217846 18822 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:39.218024 18822 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:39.220192 18822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5: Generated new TSK 0
I20260619 20:44:39.220366 18822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:44:39.227393 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46805
--builtin_ntp_servers=127.13.195.84:37353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:44:39.310138 18832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:39.310338 18832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:39.310360 18832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:39.311605 18832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:39.311643 18832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:39.311669 18832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:39.313011 18832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:39.313234 18832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:39.313541 18832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:39.318507 18837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.318533 18838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.318533 18840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.319043 18832 server_base.cc:1034] running on GCE node
I20260619 20:44:39.319278 18832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:39.319574 18832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:39.320863 18832 hybrid_clock.cc:648] HybridClock initialized: now 1781901879320848 us; error 31 us; skew 500 ppm
I20260619 20:44:39.322131 18832 webserver.cc:460] Webserver started at http://127.13.195.65:33013/ using document root <none> and password file <none>
I20260619 20:44:39.322391 18832 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:39.322497 18832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:39.322659 18832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:39.323700 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "56f615666bc945feb63725987051cd07"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "61ab575526996d137fa4c67bfc1023ad"
server_key_iv: "2d6dfe8dd011be596d5e1d6e5d5bb4e0"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.324074 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "56f615666bc945feb63725987051cd07"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "61ab575526996d137fa4c67bfc1023ad"
server_key_iv: "2d6dfe8dd011be596d5e1d6e5d5bb4e0"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.325404 18832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:39.326149 18846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.326324 18832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:39.326413 18832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "56f615666bc945feb63725987051cd07"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "61ab575526996d137fa4c67bfc1023ad"
server_key_iv: "2d6dfe8dd011be596d5e1d6e5d5bb4e0"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.326550 18832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:39.329756 18832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:39.330015 18832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:39.330153 18832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:39.330358 18832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:39.330678 18832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:39.330736 18832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.330773 18832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:39.330799 18832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.336498 18832 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:42465
I20260619 20:44:39.336591 18960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:42465 every 8 connection(s)
I20260619 20:44:39.336876 18832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:39.339818 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.337573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:42965 (local address 127.13.195.126:46805)
0619 20:44:39.337635 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.337637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.337869 (+ 232us) server_negotiation.cc:407] Connection header received
0619 20:44:39.338353 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.338354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.338360 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.338407 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.339563 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.339565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.339565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.339569 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.339583 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.339587 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.339645 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.339737 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.339751 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:44:39.340059 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.337582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:46805 (local address 127.13.195.65:42965)
0619 20:44:39.337755 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.337786 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.338253 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.338513 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.338516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.338524 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.339399 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.339399 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.339425 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.339452 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.339645 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.339651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.339654 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.339748 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.339807 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260619 20:44:39.340544 18961 heartbeater.cc:347] Connected to a master server at 127.13.195.126:46805
I20260619 20:44:39.340646 18961 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:39.340791 18961 heartbeater.cc:511] Master 127.13.195.126:46805 requested a full tablet report, sending...
I20260619 20:44:39.341286 18772 ts_manager.cc:194] Registered new tserver with Master: 56f615666bc945feb63725987051cd07 (127.13.195.65:42465)
I20260619 20:44:39.341980 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18832
I20260619 20:44:39.342060 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:39.342177 14093 external_mini_cluster.cc:1428] Setting key 4b817d7f0cb34739558eec51d63a0987
I20260619 20:44:39.343041 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46805
--builtin_ntp_servers=127.13.195.84:37353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:44:39.423182 18964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:39.423383 18964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:39.423405 18964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:39.424480 18964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:39.424515 18964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:39.424539 18964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:39.425784 18964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:39.425962 18964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:39.426200 18964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:39.431327 18972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.431368 18970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.431537 18964 server_base.cc:1034] running on GCE node
W20260619 20:44:39.431581 18969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.431782 18964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:39.432039 18964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:39.433207 18964 hybrid_clock.cc:648] HybridClock initialized: now 1781901879433200 us; error 37 us; skew 500 ppm
I20260619 20:44:39.434561 18964 webserver.cc:460] Webserver started at http://127.13.195.66:34303/ using document root <none> and password file <none>
I20260619 20:44:39.434754 18964 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:39.434794 18964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:39.434893 18964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:39.435935 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "a534b5510a844d9b90a7e8173996a826"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "56cb5aa21ad11465cf9107dd429681a3"
server_key_iv: "0de816cc6f467775f44dc7c533c6d3e8"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.436225 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "a534b5510a844d9b90a7e8173996a826"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "56cb5aa21ad11465cf9107dd429681a3"
server_key_iv: "0de816cc6f467775f44dc7c533c6d3e8"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.437507 18964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:39.438166 18978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.438330 18964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:39.438434 18964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "a534b5510a844d9b90a7e8173996a826"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "56cb5aa21ad11465cf9107dd429681a3"
server_key_iv: "0de816cc6f467775f44dc7c533c6d3e8"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.438539 18964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:39.441792 18964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:39.442057 18964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:39.442193 18964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:39.442414 18964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:39.442749 18964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:39.442803 18964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.442842 18964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:39.442868 18964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.448639 18964 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:46861
I20260619 20:44:39.448673 19092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:46861 every 8 connection(s)
I20260619 20:44:39.448992 18964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:39.452009 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.449670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:42319 (local address 127.13.195.126:46805)
0619 20:44:39.449734 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.449736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.449995 (+ 259us) server_negotiation.cc:407] Connection header received
0619 20:44:39.450547 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.450549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.450555 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.450620 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.451739 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.451740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.451741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.451745 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.451759 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.451763 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.451815 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.451916 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.451931 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:44:39.452196 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.449659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:46805 (local address 127.13.195.66:42319)
0619 20:44:39.449860 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.449903 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.450384 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.450691 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.450695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.450704 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.451586 (+ 882us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.451586 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.451613 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.451640 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.451825 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.451830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.451833 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.451927 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.451986 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260619 20:44:39.452715 19093 heartbeater.cc:347] Connected to a master server at 127.13.195.126:46805
I20260619 20:44:39.452787 19093 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:39.452943 19093 heartbeater.cc:511] Master 127.13.195.126:46805 requested a full tablet report, sending...
I20260619 20:44:39.453326 18772 ts_manager.cc:194] Registered new tserver with Master: a534b5510a844d9b90a7e8173996a826 (127.13.195.66:46861)
I20260619 20:44:39.457453 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 18964
I20260619 20:44:39.457541 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:39.457675 14093 external_mini_cluster.cc:1428] Setting key 7ce1708830fb3e4fe5bb2df768bcab89
I20260619 20:44:39.458603 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46805
--builtin_ntp_servers=127.13.195.84:37353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:44:39.540825 19096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:39.541074 19096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:39.541112 19096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:39.542354 19096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:39.542415 19096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:39.542477 19096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:39.543836 19096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:46805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:39.544099 19096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:39.544404 19096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:39.549674 19102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.549701 19096 server_base.cc:1034] running on GCE node
W20260619 20:44:39.549669 19104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.549700 19101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.550114 19096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:39.550354 19096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:39.551502 19096 hybrid_clock.cc:648] HybridClock initialized: now 1781901879551471 us; error 39 us; skew 500 ppm
I20260619 20:44:39.552817 19096 webserver.cc:460] Webserver started at http://127.13.195.67:43069/ using document root <none> and password file <none>
I20260619 20:44:39.553063 19096 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:39.553108 19096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:39.553269 19096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:39.554311 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "4c38d314473d40998cc9aa3a835e3616"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "cb6e36a6d3b5ae8ca40e1bcb21cdde8a"
server_key_iv: "fc7c3e514a318a578fa75ee06a1ae28a"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.554724 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "4c38d314473d40998cc9aa3a835e3616"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "cb6e36a6d3b5ae8ca40e1bcb21cdde8a"
server_key_iv: "fc7c3e514a318a578fa75ee06a1ae28a"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.555972 19096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:39.556640 19110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.556823 19096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:39.556907 19096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "4c38d314473d40998cc9aa3a835e3616"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "cb6e36a6d3b5ae8ca40e1bcb21cdde8a"
server_key_iv: "fc7c3e514a318a578fa75ee06a1ae28a"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.556977 19096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:39.560320 19096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:39.560572 19096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:39.560698 19096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:39.560899 19096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:39.561192 19096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:39.561228 19096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.561277 19096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:39.561302 19096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.567142 19096 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:44695
I20260619 20:44:39.567178 19224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:44695 every 8 connection(s)
I20260619 20:44:39.567504 19096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:39.570261 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.568210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:59989 (local address 127.13.195.126:46805)
0619 20:44:39.568276 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.568279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.568501 (+ 222us) server_negotiation.cc:407] Connection header received
0619 20:44:39.568922 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.568923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.568928 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.568991 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.570079 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.570080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.570081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.570084 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.570098 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.570102 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.570144 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.570195 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.570207 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:44:39.570436 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.568198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:46805 (local address 127.13.195.67:59989)
0619 20:44:39.568383 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.568414 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.568841 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.569045 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.569049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.569056 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.569933 (+ 877us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.569934 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.569971 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.569998 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.570149 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.570151 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.570154 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.570202 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.570246 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260619 20:44:39.570902 19225 heartbeater.cc:347] Connected to a master server at 127.13.195.126:46805
I20260619 20:44:39.571004 19225 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:39.571164 19225 heartbeater.cc:511] Master 127.13.195.126:46805 requested a full tablet report, sending...
I20260619 20:44:39.571520 18772 ts_manager.cc:194] Registered new tserver with Master: 4c38d314473d40998cc9aa3a835e3616 (127.13.195.67:44695)
I20260619 20:44:39.572935 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19096
I20260619 20:44:39.573045 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:39.573212 14093 external_mini_cluster.cc:1428] Setting key e1441c8cf99f84a68e2431e10be7f4a0
I20260619 20:44:39.574182 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:44:39.576434 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.575501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36118 (local address 127.13.195.126:46805)
0619 20:44:39.575539 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.575541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.575685 (+ 144us) server_negotiation.cc:407] Connection header received
0619 20:44:39.575736 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.575737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.575741 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.575790 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.576273 (+ 483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.576274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.576274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.576278 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.576290 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.576293 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.576336 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.576375 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.576386 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:39.578001 18772 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:39.579576 18772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:39.583593 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.581857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:44695 (local address 127.0.0.1:50722)
0619 20:44:39.581972 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.581981 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.582022 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.582550 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.582557 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.582563 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.583142 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.583142 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.583156 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.583174 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.583412 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.583418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.583421 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.583498 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.583512 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
I20260619 20:44:39.583630 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.582196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.13.195.67:44695)
0619 20:44:39.582389 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.582393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.582403 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:39.582450 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.582454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.582463 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.582548 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.583257 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.583264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.583265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.583269 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.583305 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.583328 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.583414 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.583521 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.583538 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260619 20:44:39.583833 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.582020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:46861 (local address 127.0.0.1:54322)
0619 20:44:39.582301 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.582310 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.582358 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.582871 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.582872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.582874 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.583278 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.583279 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.583287 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.583301 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.583510 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.583511 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.583511 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.583564 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.583577 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":57,"threads_started":1}
I20260619 20:44:39.583860 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.582138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.13.195.66:46861)
0619 20:44:39.582275 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.582280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.582354 (+ 74us) server_negotiation.cc:407] Connection header received
0619 20:44:39.582469 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.582473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.582483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.582555 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.583373 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.583379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.583380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.583384 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.583418 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.583445 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.583515 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.583635 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.583780 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260619 20:44:39.583943 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.581636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:42465 (local address 127.0.0.1:48420)
0619 20:44:39.581758 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.581789 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.581857 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.582624 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.582624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.582627 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.583561 (+ 934us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.583562 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.583567 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.583576 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.583817 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.583818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.583819 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.583861 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.583874 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":84,"threads_started":1}
I20260619 20:44:39.584194 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.581741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.13.195.65:42465)
0619 20:44:39.581915 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.581919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.581929 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:39.581953 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.581955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.581965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.582062 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.583659 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.583667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.583667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.583671 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.583706 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.583738 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.583793 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.584089 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.584112 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260619 20:44:39.584473 19159 tablet_service.cc:1467] Processing CreateTablet for tablet f056f3d7463347aaa9d867f572ee6aee (DEFAULT_TABLE table=test-table [id=ada85c6923a2457d956bdf4dc6f6b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:39.584551 19027 tablet_service.cc:1467] Processing CreateTablet for tablet f056f3d7463347aaa9d867f572ee6aee (DEFAULT_TABLE table=test-table [id=ada85c6923a2457d956bdf4dc6f6b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:39.584811 19027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f056f3d7463347aaa9d867f572ee6aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:39.584829 19159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f056f3d7463347aaa9d867f572ee6aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:39.584786 18895 tablet_service.cc:1467] Processing CreateTablet for tablet f056f3d7463347aaa9d867f572ee6aee (DEFAULT_TABLE table=test-table [id=ada85c6923a2457d956bdf4dc6f6b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:39.585088 18895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f056f3d7463347aaa9d867f572ee6aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:39.587497 19239 tablet_bootstrap.cc:492] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Bootstrap starting.
I20260619 20:44:39.588018 19239 tablet_bootstrap.cc:654] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:39.588129 19240 tablet_bootstrap.cc:492] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07: Bootstrap starting.
I20260619 20:44:39.588349 19239 log.cc:824] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:39.588747 19240 tablet_bootstrap.cc:654] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:39.588752 19241 tablet_bootstrap.cc:492] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Bootstrap starting.
I20260619 20:44:39.589074 19240 log.cc:824] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:39.589195 19239 tablet_bootstrap.cc:492] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: No bootstrap required, opened a new log
I20260619 20:44:39.589268 19239 ts_tablet_manager.cc:1397] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:39.589437 19241 tablet_bootstrap.cc:654] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:39.589746 19241 log.cc:824] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:39.590404 19241 tablet_bootstrap.cc:492] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: No bootstrap required, opened a new log
I20260619 20:44:39.590507 19241 ts_tablet_manager.cc:1397] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:39.591074 19240 tablet_bootstrap.cc:492] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07: No bootstrap required, opened a new log
I20260619 20:44:39.591149 19240 ts_tablet_manager.cc:1397] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:44:39.591334 19239 raft_consensus.cc:348] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.591514 19239 raft_consensus.cc:374] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:39.591549 19239 raft_consensus.cc:729] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a534b5510a844d9b90a7e8173996a826, State: Initialized, Role: FOLLOWER
I20260619 20:44:39.591687 19239 consensus_queue.cc:260] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.591943 19239 ts_tablet_manager.cc:1428] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:44:39.592087 19093 heartbeater.cc:503] Master 127.13.195.126:46805 was elected leader, sending a full tablet report...
I20260619 20:44:39.593142 19241 raft_consensus.cc:348] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.593257 19241 raft_consensus.cc:374] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:39.593278 19241 raft_consensus.cc:729] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c38d314473d40998cc9aa3a835e3616, State: Initialized, Role: FOLLOWER
I20260619 20:44:39.593384 19241 consensus_queue.cc:260] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.593640 19241 ts_tablet_manager.cc:1428] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:44:39.593659 19225 heartbeater.cc:503] Master 127.13.195.126:46805 was elected leader, sending a full tablet report...
I20260619 20:44:39.593668 19240 raft_consensus.cc:348] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.593842 19240 raft_consensus.cc:374] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:39.593889 19240 raft_consensus.cc:729] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f615666bc945feb63725987051cd07, State: Initialized, Role: FOLLOWER
I20260619 20:44:39.594008 19240 consensus_queue.cc:260] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.594277 19240 ts_tablet_manager.cc:1428] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:44:39.594300 18961 heartbeater.cc:503] Master 127.13.195.126:46805 was elected leader, sending a full tablet report...
I20260619 20:44:39.599275 19247 raft_consensus.cc:482] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:39.599376 19247 raft_consensus.cc:504] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.599705 19247 leader_election.cc:288] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c38d314473d40998cc9aa3a835e3616 (127.13.195.67:44695), a534b5510a844d9b90a7e8173996a826 (127.13.195.66:46861)
I20260619 20:44:39.601090 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.599856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:44695 (local address 127.13.195.65:58515)
0619 20:44:39.600064 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.600076 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.600120 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.600274 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.600275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.600278 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.600734 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.600735 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.600744 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.600758 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.600941 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.600942 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.600943 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.600991 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.601006 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260619 20:44:39.601373 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.599921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:58515 (local address 127.13.195.67:44695)
0619 20:44:39.600015 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.600018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.600175 (+ 157us) server_negotiation.cc:407] Connection header received
0619 20:44:39.600197 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.600198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.600203 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.600251 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.600834 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.600835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.600836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.600840 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.600859 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.600865 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.600919 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.601269 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.601284 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 20:44:39.601553 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.600055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:46017 (local address 127.13.195.66:46861)
0619 20:44:39.600094 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.600096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.600150 (+ 54us) server_negotiation.cc:407] Connection header received
0619 20:44:39.600243 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.600244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.600249 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.600278 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.601085 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.601086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.601086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.601091 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.601103 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.601107 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.601211 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.601411 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.601453 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:44:39.601622 19179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f056f3d7463347aaa9d867f572ee6aee" candidate_uuid: "56f615666bc945feb63725987051cd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c38d314473d40998cc9aa3a835e3616" is_pre_election: true
I20260619 20:44:39.601748 19179 raft_consensus.cc:2454] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f615666bc945feb63725987051cd07 in term 0.
I20260619 20:44:39.601760 19248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.599948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:46861 (local address 127.13.195.65:46017)
0619 20:44:39.600080 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.600088 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.600170 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.600343 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.600344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.600346 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:39.600977 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:44:39.600978 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:39.600984 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:44:39.601003 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:39.601326 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:39.601326 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:39.601327 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:39.601405 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:44:39.601543 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260619 20:44:39.602005 18851 leader_election.cc:302] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c38d314473d40998cc9aa3a835e3616, 56f615666bc945feb63725987051cd07; no voters:
I20260619 20:44:39.602229 19247 raft_consensus.cc:2781] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:39.602298 19247 raft_consensus.cc:482] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:39.602322 19247 raft_consensus.cc:3037] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.602505 19047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f056f3d7463347aaa9d867f572ee6aee" candidate_uuid: "56f615666bc945feb63725987051cd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a534b5510a844d9b90a7e8173996a826" is_pre_election: true
I20260619 20:44:39.602625 19047 raft_consensus.cc:2454] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f615666bc945feb63725987051cd07 in term 0.
I20260619 20:44:39.603269 19247 raft_consensus.cc:504] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.603395 19247 leader_election.cc:288] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [CANDIDATE]: Term 1 election: Requested vote from peers 4c38d314473d40998cc9aa3a835e3616 (127.13.195.67:44695), a534b5510a844d9b90a7e8173996a826 (127.13.195.66:46861)
I20260619 20:44:39.603542 19179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f056f3d7463347aaa9d867f572ee6aee" candidate_uuid: "56f615666bc945feb63725987051cd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c38d314473d40998cc9aa3a835e3616"
I20260619 20:44:39.603601 19047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f056f3d7463347aaa9d867f572ee6aee" candidate_uuid: "56f615666bc945feb63725987051cd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a534b5510a844d9b90a7e8173996a826"
I20260619 20:44:39.603622 19179 raft_consensus.cc:3037] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.603679 19047 raft_consensus.cc:3037] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.604501 19179 raft_consensus.cc:2454] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f615666bc945feb63725987051cd07 in term 1.
I20260619 20:44:39.604629 19047 raft_consensus.cc:2454] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f615666bc945feb63725987051cd07 in term 1.
I20260619 20:44:39.604673 18851 leader_election.cc:302] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c38d314473d40998cc9aa3a835e3616, 56f615666bc945feb63725987051cd07; no voters:
I20260619 20:44:39.604784 19247 raft_consensus.cc:2781] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:39.604926 19247 raft_consensus.cc:686] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 1 LEADER]: Becoming Leader. State: Replica: 56f615666bc945feb63725987051cd07, State: Running, Role: LEADER
I20260619 20:44:39.605017 19247 consensus_queue.cc:237] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } }
I20260619 20:44:39.605726 18771 catalog_manager.cc:5495] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56f615666bc945feb63725987051cd07 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "56f615666bc945feb63725987051cd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56f615666bc945feb63725987051cd07" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:39.610510 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.609576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.13.195.65:42465)
0619 20:44:39.609625 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.609627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.609635 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:39.609663 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.609664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.609681 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.609731 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.610300 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.610301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.610301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.610304 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.610314 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.610317 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.610358 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.610400 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.610411 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:44:39.612022 19179 raft_consensus.cc:1261] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 1 FOLLOWER]: Refusing update from remote peer 56f615666bc945feb63725987051cd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:39.612049 19047 raft_consensus.cc:1261] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 1 FOLLOWER]: Refusing update from remote peer 56f615666bc945feb63725987051cd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:39.612341 19247 consensus_queue.cc:1035] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c38d314473d40998cc9aa3a835e3616" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:39.612468 19249 consensus_queue.cc:1035] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a534b5510a844d9b90a7e8173996a826" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:39.615382 19253 mvcc.cc:205] Tried to move back new op lower bound from 7298670098888351744 to 7298670098862747648. Current Snapshot: MvccSnapshot[applied={T|T < 7298670098888351744 or (T in {7298670098888351744})}]
I20260619 20:44:39.615559 19254 mvcc.cc:205] Tried to move back new op lower bound from 7298670098888351744 to 7298670098862747648. Current Snapshot: MvccSnapshot[applied={T|T < 7298670098888351744}]
I20260619 20:44:39.617097 18772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:39.617187 18772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:39.617851 18772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec8ed09abf414576a1e62386426d4ff5: Sending DeleteTablet for 3 replicas of tablet f056f3d7463347aaa9d867f572ee6aee
I20260619 20:44:39.618148 18895 tablet_service.cc:1514] Processing DeleteTablet for tablet f056f3d7463347aaa9d867f572ee6aee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:39 UTC) from {username='slave'} at 127.0.0.1:48420
I20260619 20:44:39.618197 19027 tablet_service.cc:1514] Processing DeleteTablet for tablet f056f3d7463347aaa9d867f572ee6aee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:39 UTC) from {username='slave'} at 127.0.0.1:54322
I20260619 20:44:39.618294 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18832
I20260619 20:44:39.618368 19262 tablet_replica.cc:330] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07: stopping tablet replica
I20260619 20:44:39.618384 19159 tablet_service.cc:1514] Processing DeleteTablet for tablet f056f3d7463347aaa9d867f572ee6aee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:39 UTC) from {username='slave'} at 127.0.0.1:50722
I20260619 20:44:39.618501 19262 raft_consensus.cc:2229] T f056f3d7463347aaa9d867f572ee6aee P 56f615666bc945feb63725987051cd07 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:39.618618 19263 tablet_replica.cc:330] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: stopping tablet replica
I20260619 20:44:39.619539 19263 raft_consensus.cc:2229] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:39.619688 19263 raft_consensus.cc:2258] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:39.620155 19263 ts_tablet_manager.cc:1905] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:39.622121 19263 ts_tablet_manager.cc:1918] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:39.622200 19263 log.cc:1196] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/f056f3d7463347aaa9d867f572ee6aee
I20260619 20:44:39.622484 19263 ts_tablet_manager.cc:1939] T f056f3d7463347aaa9d867f572ee6aee P a534b5510a844d9b90a7e8173996a826: Deleting consensus metadata
I20260619 20:44:39.622562 19264 tablet_replica.cc:330] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: stopping tablet replica
I20260619 20:44:39.622668 19264 raft_consensus.cc:2229] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:39.622764 19264 raft_consensus.cc:2258] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:39.622872 18759 catalog_manager.cc:4841] TS a534b5510a844d9b90a7e8173996a826 (127.13.195.66:46861): tablet f056f3d7463347aaa9d867f572ee6aee (table test-table [id=ada85c6923a2457d956bdf4dc6f6b922]) successfully deleted
I20260619 20:44:39.623145 19264 ts_tablet_manager.cc:1905] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 20:44:39.624356 18759 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:42465 (error 108)
I20260619 20:44:39.624368 19264 ts_tablet_manager.cc:1918] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:39.624451 19264 log.cc:1196] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/f056f3d7463347aaa9d867f572ee6aee
I20260619 20:44:39.624730 19264 ts_tablet_manager.cc:1939] T f056f3d7463347aaa9d867f572ee6aee P 4c38d314473d40998cc9aa3a835e3616: Deleting consensus metadata
I20260619 20:44:39.624753 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18964
I20260619 20:44:39.624863 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.624718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:42465 (local address 127.0.0.1:48442)
0619 20:44:39.624784 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.624826 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 20:44:39.625521 18759 catalog_manager.cc:4584] TS 56f615666bc945feb63725987051cd07 (127.13.195.65:42465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f056f3d7463347aaa9d867f572ee6aee: Network error: Client connection negotiation failed: client connection to 127.13.195.65:42465: connect: Connection refused (error 111)
I20260619 20:44:39.625649 18759 catalog_manager.cc:4841] TS 4c38d314473d40998cc9aa3a835e3616 (127.13.195.67:44695): tablet f056f3d7463347aaa9d867f572ee6aee (table test-table [id=ada85c6923a2457d956bdf4dc6f6b922]) successfully deleted
I20260619 20:44:39.629505 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19096
I20260619 20:44:39.634194 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 18740
2026-06-19T20:44:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (565 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 20:44:39.653671 14093 mini_oidc.cc:152] Starting JWKS server
I20260619 20:44:39.654832 14093 webserver.cc:460] Webserver started at https://127.0.0.1:45221/ using document root <none> and password file <none>
I20260619 20:44:39.654913 14093 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:44:39.655052 14093 webserver.cc:460] Webserver started at http://127.0.0.1:33903/ using document root <none> and password file <none>
2026-06-19T20:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:39Z Disabled control of system clock
I20260619 20:44:39.662575 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42525
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:42525
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45221/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/testchainca.cert with env {}
W20260619 20:44:39.742530 19271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:39.742736 19271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:39.742758 19271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:39.743845 19271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:39.743881 19271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:39.743893 19271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:39.743906 19271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:39.743918 19271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:44:39.743930 19271 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:44:39.743942 19271 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45221/jwks.json
I20260619 20:44:39.745452 19271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:42525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45221/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:39.745648 19271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:39.745810 19271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:39.750625 19276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.750687 19277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.750644 19279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.750861 19271 server_base.cc:1034] running on GCE node
I20260619 20:44:39.751101 19271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:39.751391 19271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:39.752574 19271 hybrid_clock.cc:648] HybridClock initialized: now 1781901879752557 us; error 27 us; skew 500 ppm
I20260619 20:44:39.753772 19271 webserver.cc:460] Webserver started at http://127.13.195.126:44973/ using document root <none> and password file <none>
I20260619 20:44:39.754022 19271 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:39.754094 19271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:39.754246 19271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:39.755366 19271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "3b8ce466b5ce4f50ba66f86d4da495d8"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "f68a202349ec31fa55be801688780800"
server_key_iv: "4efa3f3db55870d4a347a66c2cee479c"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.755735 19271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "3b8ce466b5ce4f50ba66f86d4da495d8"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "f68a202349ec31fa55be801688780800"
server_key_iv: "4efa3f3db55870d4a347a66c2cee479c"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.757030 19271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:39.757745 19285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.757920 19271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:39.758020 19271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "3b8ce466b5ce4f50ba66f86d4da495d8"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "f68a202349ec31fa55be801688780800"
server_key_iv: "4efa3f3db55870d4a347a66c2cee479c"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.758104 19271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:39.780014 19271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:39.780340 19271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:39.780498 19271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:39.784637 19271 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:42525
I20260619 20:44:39.784674 19338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:42525 every 8 connection(s)
I20260619 20:44:39.785022 19271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:39.785938 19339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:39.786787 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19271
I20260619 20:44:39.786886 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:39.787084 14093 external_mini_cluster.cc:1428] Setting key dca00a0963c61bd07f94aa3ca252222a
I20260619 20:44:39.789335 19339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8: Bootstrap starting.
I20260619 20:44:39.790050 19339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:39.790342 19339 log.cc:824] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:39.791095 19339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8: No bootstrap required, opened a new log
I20260619 20:44:39.793061 19339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42525 } }
I20260619 20:44:39.793205 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.787614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.13.195.126:42525)
0619 20:44:39.787832 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.787842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.787875 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:44:39.788462 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.788471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.788563 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.788713 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:39.788932 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.790277 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.791302 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.791708 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.791852 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.792572 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:39.792576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:39.792576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:39.792734 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:39.792765 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:39.792775 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:39.792852 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:39.792935 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 20:44:39.793011 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260619 20:44:39.793268 19339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:39.793336 19339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8ce466b5ce4f50ba66f86d4da495d8, State: Initialized, Role: FOLLOWER
I20260619 20:44:39.793490 19339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42525 } }
I20260619 20:44:39.793587 19339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:39.793632 19339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:39.793679 19339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:39.794270 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:44:39.794373 19339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42525 } }
I20260619 20:44:39.794518 19339 leader_election.cc:302] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b8ce466b5ce4f50ba66f86d4da495d8; no voters:
I20260619 20:44:39.794713 19339 leader_election.cc:288] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:39.794747 19344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:39.794852 19344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b8ce466b5ce4f50ba66f86d4da495d8, State: Running, Role: LEADER
I20260619 20:44:39.794935 19344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42525 } }
I20260619 20:44:39.794974 19339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:39.795334 19344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b8ce466b5ce4f50ba66f86d4da495d8. Latest consensus state: current_term: 1 leader_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42525 } } }
I20260619 20:44:39.795418 19344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:39.795424 19346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8ce466b5ce4f50ba66f86d4da495d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42525 } } }
I20260619 20:44:39.795523 19346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:39.795667 19350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:39.796162 19350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:39.797758 19350 catalog_manager.cc:1347] Generated new cluster ID: 883347665052451fa5d68cbbe2b4ca8f
I20260619 20:44:39.797807 19350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:39.808152 19350 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:39.808830 19350 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:39.812702 19350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b8ce466b5ce4f50ba66f86d4da495d8: Generated new TSK 0
I20260619 20:44:39.812879 19350 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 20:44:39.837807 19370 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 20:44:39.837917 19370 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 20:44:39.838099 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.823054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.13.195.126:42525)
0619 20:44:39.823123 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.823124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.823226 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:44:39.823324 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.823325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.823334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.823387 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:44:39.823524 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.824039 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.825163 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.825351 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.825399 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.825670 (+ 271us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:44:39.837850 (+ 12180us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 20:44:39.838026 (+ 176us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45852: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 20:44:39.838168 19342 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45852: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 20:44:39.838542 19368 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:39.822943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42525 (local address 127.0.0.1:45852)
0619 20:44:39.823150 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:39.823162 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:39.823234 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:39.823400 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:39.823400 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:39.823402 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:44:39.823467 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:39.823469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.824117 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.824118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:39.825081 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:39.825085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:39.825445 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:39.825446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:39.825513 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:39.825536 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:44:39.838094 (+ 12558us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 20:44:39.838206 (+ 112us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.195.126:42525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.195.126:42525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 20:44:39.838932 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19271
2026-06-19T20:44:39Z chronyd exiting
I20260619 20:44:39.874576 14093 test_util.cc:175] -----------------------------------------------
I20260619 20:44:39.874650 14093 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901857247738-14093-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 20:44:39.875406 14093 mini_oidc.cc:152] Starting JWKS server
I20260619 20:44:39.876093 14093 webserver.cc:460] Webserver started at https://127.0.0.1:37845/ using document root <none> and password file <none>
I20260619 20:44:39.876163 14093 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:44:39.876330 14093 webserver.cc:460] Webserver started at http://127.0.0.1:44127/ using document root <none> and password file <none>
2026-06-19T20:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:39Z Disabled control of system clock
I20260619 20:44:39.883963 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42355
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:42355
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37845/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/testchainca.cert with env {}
W20260619 20:44:39.967186 19376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:39.967419 19376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:39.967447 19376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:39.968807 19376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:39.968856 19376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:39.968873 19376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:39.968925 19376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:39.968958 19376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:44:39.968987 19376 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:44:39.969026 19376 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37845/jwks.json
I20260619 20:44:39.970772 19376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:42355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37845/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:39.971045 19376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:39.971306 19376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:39.976890 19382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.976867 19384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:39.976861 19381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:39.977106 19376 server_base.cc:1034] running on GCE node
I20260619 20:44:39.977519 19376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:39.977859 19376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:39.979060 19376 hybrid_clock.cc:648] HybridClock initialized: now 1781901879979044 us; error 25 us; skew 500 ppm
I20260619 20:44:39.980239 19376 webserver.cc:460] Webserver started at http://127.13.195.126:39661/ using document root <none> and password file <none>
I20260619 20:44:39.980500 19376 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:39.980571 19376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:39.980718 19376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:39.981747 19376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "52635cbedb1b4fe888145150bac6bc4a"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "7245be5442ca280961062d93a3023669"
server_key_iv: "a11162a4045663c7ff2931bda56a68e9"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.982105 19376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "52635cbedb1b4fe888145150bac6bc4a"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "7245be5442ca280961062d93a3023669"
server_key_iv: "a11162a4045663c7ff2931bda56a68e9"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.983453 19376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:39.984145 19390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:39.984315 19376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:39.984416 19376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "52635cbedb1b4fe888145150bac6bc4a"
format_stamp: "Formatted at 2026-06-19 20:44:39 on dist-test-slave-n2td"
server_key: "7245be5442ca280961062d93a3023669"
server_key_iv: "a11162a4045663c7ff2931bda56a68e9"
server_key_version: "encryptionkey@0"
I20260619 20:44:39.984504 19376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:39.991015 19376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:39.991294 19376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:39.991438 19376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:39.995289 19376 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:42355
I20260619 20:44:39.995318 19442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:42355 every 8 connection(s)
I20260619 20:44:39.995652 19376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:39.996275 19443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:39.998011 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19376
I20260619 20:44:39.998124 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:39.998311 14093 external_mini_cluster.cc:1428] Setting key 586f947e68e002234b2c07b989281c43
I20260619 20:44:39.999351 19443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a: Bootstrap starting.
I20260619 20:44:40.000046 19443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:40.000360 19443 log.cc:824] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:40.001093 19443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a: No bootstrap required, opened a new log
I20260619 20:44:40.003268 19443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52635cbedb1b4fe888145150bac6bc4a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42355 } }
I20260619 20:44:40.003397 19443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:40.003419 19443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52635cbedb1b4fe888145150bac6bc4a, State: Initialized, Role: FOLLOWER
I20260619 20:44:40.003561 19443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52635cbedb1b4fe888145150bac6bc4a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42355 } }
I20260619 20:44:40.003638 19443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:40.003659 19443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:40.003726 19443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:40.004307 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:39.998828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.13.195.126:42355)
0619 20:44:39.999038 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:39.999047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:39.999079 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:44:39.999605 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:39.999614 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:39.999703 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:39.999853 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:40.000019 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.001308 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.002076 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.002757 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.002907 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.003563 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:40.003568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:40.003568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:40.003800 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:40.003847 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:40.003858 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:40.003907 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:40.004022 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 20:44:40.004114 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260619 20:44:40.004446 19443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52635cbedb1b4fe888145150bac6bc4a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42355 } }
I20260619 20:44:40.004557 19443 leader_election.cc:302] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52635cbedb1b4fe888145150bac6bc4a; no voters:
I20260619 20:44:40.004727 19443 leader_election.cc:288] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:40.004755 19448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:40.004906 19448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [term 1 LEADER]: Becoming Leader. State: Replica: 52635cbedb1b4fe888145150bac6bc4a, State: Running, Role: LEADER
I20260619 20:44:40.004976 19443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:40.004999 19448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52635cbedb1b4fe888145150bac6bc4a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42355 } }
I20260619 20:44:40.005321 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:44:40.005336 19450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52635cbedb1b4fe888145150bac6bc4a. Latest consensus state: current_term: 1 leader_uuid: "52635cbedb1b4fe888145150bac6bc4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52635cbedb1b4fe888145150bac6bc4a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42355 } } }
I20260619 20:44:40.005414 19450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:40.005633 19455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:40.006122 19455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:40.006477 19449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52635cbedb1b4fe888145150bac6bc4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52635cbedb1b4fe888145150bac6bc4a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42355 } } }
I20260619 20:44:40.006776 19449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:40.008391 19455 catalog_manager.cc:1347] Generated new cluster ID: 729be9f644e340ed8d26ad42124e13d4
I20260619 20:44:40.008440 19455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:40.020226 19455 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:40.020768 19455 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:40.027308 19455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52635cbedb1b4fe888145150bac6bc4a: Generated new TSK 0
I20260619 20:44:40.027524 19455 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 20:44:40.047575 19474 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 20:44:40.047675 19474 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 20:44:40.048039 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:40.033535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53808 (local address 127.13.195.126:42355)
0619 20:44:40.033591 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:40.033593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:40.033695 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:44:40.033741 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:40.033742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:40.033755 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:40.033836 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:44:40.033949 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.034758 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.035501 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.035822 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.035894 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.036070 (+ 176us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:44:40.047800 (+ 11730us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37845/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 20:44:40.047962 (+ 162us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37845/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 20:44:40.048116 19446 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37845/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 20:44:40.048161 19473 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:40.033448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42355 (local address 127.0.0.1:53808)
0619 20:44:40.033619 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:40.033638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:40.033691 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:40.033839 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:40.033840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:40.033842 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:44:40.033888 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:40.033890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.034831 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.034832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:40.035414 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:40.035416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.035899 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.035900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:40.035939 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.035943 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:44:40.047926 (+ 11983us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37845/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 20:44:40.048041 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.195.126:42355: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37845/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260619 20:44:40.048486 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19376
2026-06-19T20:44:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (210 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 20:44:40.086223 14093 mini_oidc.cc:152] Starting JWKS server
I20260619 20:44:40.086995 14093 webserver.cc:460] Webserver started at https://127.0.0.1:44899/ using document root <none> and password file <none>
I20260619 20:44:40.087065 14093 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:44:40.087229 14093 webserver.cc:460] Webserver started at http://127.0.0.1:33521/ using document root <none> and password file <none>
2026-06-19T20:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:40Z Disabled control of system clock
I20260619 20:44:40.094791 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:46815
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:38671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:46815
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44899/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 20:44:40.174372 19480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:40.174602 19480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:40.174624 19480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:40.175709 19480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:40.175741 19480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:40.175755 19480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:40.175766 19480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:40.175782 19480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:44:40.175796 19480 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:44:40.175806 19480 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44899/jwks.json
I20260619 20:44:40.177073 19480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:46815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:46815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44899/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:40.177258 19480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:40.177495 19480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:40.182123 19485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:40.182124 19488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:40.182145 19486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:40.182367 19480 server_base.cc:1034] running on GCE node
I20260619 20:44:40.182605 19480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:40.182838 19480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:40.183976 19480 hybrid_clock.cc:648] HybridClock initialized: now 1781901880183963 us; error 28 us; skew 500 ppm
I20260619 20:44:40.185237 19480 webserver.cc:460] Webserver started at http://127.13.195.126:46169/ using document root <none> and password file <none>
I20260619 20:44:40.185495 19480 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:40.185544 19480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:40.185719 19480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:40.186866 19480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "33a52c6541b742c0bb9c5ce1e48f2969"
format_stamp: "Formatted at 2026-06-19 20:44:40 on dist-test-slave-n2td"
server_key: "fe6998a6b345789d210db8c0bfd407c8"
server_key_iv: "7c70f875727a4f1a345c1b2874421490"
server_key_version: "encryptionkey@0"
I20260619 20:44:40.187220 19480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "33a52c6541b742c0bb9c5ce1e48f2969"
format_stamp: "Formatted at 2026-06-19 20:44:40 on dist-test-slave-n2td"
server_key: "fe6998a6b345789d210db8c0bfd407c8"
server_key_iv: "7c70f875727a4f1a345c1b2874421490"
server_key_version: "encryptionkey@0"
I20260619 20:44:40.188513 19480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:40.189189 19494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:40.189363 19480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:40.189450 19480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "33a52c6541b742c0bb9c5ce1e48f2969"
format_stamp: "Formatted at 2026-06-19 20:44:40 on dist-test-slave-n2td"
server_key: "fe6998a6b345789d210db8c0bfd407c8"
server_key_iv: "7c70f875727a4f1a345c1b2874421490"
server_key_version: "encryptionkey@0"
I20260619 20:44:40.189524 19480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:40.216132 19480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:40.216500 19480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:40.216671 19480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:40.220659 19480 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:46815
I20260619 20:44:40.220728 19546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:46815 every 8 connection(s)
I20260619 20:44:40.221077 19480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:40.221771 19547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:40.224553 19547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969: Bootstrap starting.
I20260619 20:44:40.225143 19547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:40.225432 19547 log.cc:824] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:40.226111 19547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969: No bootstrap required, opened a new log
I20260619 20:44:40.227731 19547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46815 } }
I20260619 20:44:40.227867 19547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:40.227890 19547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a52c6541b742c0bb9c5ce1e48f2969, State: Initialized, Role: FOLLOWER
I20260619 20:44:40.228109 19547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46815 } }
I20260619 20:44:40.228191 19547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:40.228240 19547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:40.228291 19547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:40.228820 19547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46815 } }
I20260619 20:44:40.228927 19547 leader_election.cc:302] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33a52c6541b742c0bb9c5ce1e48f2969; no voters:
I20260619 20:44:40.229146 19547 leader_election.cc:288] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:40.229169 19550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:40.229159 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19480
I20260619 20:44:40.229266 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:40.229272 19550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [term 1 LEADER]: Becoming Leader. State: Replica: 33a52c6541b742c0bb9c5ce1e48f2969, State: Running, Role: LEADER
I20260619 20:44:40.229384 19550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46815 } }
I20260619 20:44:40.229446 14093 external_mini_cluster.cc:1428] Setting key d443b28c996f52b70b2792ea95fe2de2
I20260619 20:44:40.229399 19547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:40.231082 19551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33a52c6541b742c0bb9c5ce1e48f2969. Latest consensus state: current_term: 1 leader_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46815 } } }
I20260619 20:44:40.231174 19551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:40.231297 19552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a52c6541b742c0bb9c5ce1e48f2969" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 46815 } } }
I20260619 20:44:40.231348 19552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969 [sys.catalog]: This master's current role is: LEADER
W20260619 20:44:40.231820 19567 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:44:40.231889 19567 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:44:40.232008 19566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:40.232180 19566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:40.233882 19566 catalog_manager.cc:1347] Generated new cluster ID: 2606dc28db7541d29732709f75bf24f9
I20260619 20:44:40.233947 19566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:40.235998 19556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:40.229939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46492 (local address 127.13.195.126:46815)
0619 20:44:40.230136 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:40.230146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:40.230186 (+ 40us) server_negotiation.cc:407] Connection header received
0619 20:44:40.230312 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:40.230319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:40.230389 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:40.230567 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:40.230884 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.232892 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.233856 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.234463 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.234594 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.235259 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:40.235263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:40.235264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:40.235469 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:40.235522 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:40.235537 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:40.235615 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:40.235709 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:44:40.235789 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260619 20:44:40.236878 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:44:40.243340 19566 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:40.243863 19566 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:40.249009 19566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33a52c6541b742c0bb9c5ce1e48f2969: Generated new TSK 0
I20260619 20:44:40.249189 19566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:44:40.259572 19556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:40.255917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.13.195.126:46815)
0619 20:44:40.255962 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:40.255965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:40.256066 (+ 101us) server_negotiation.cc:407] Connection header received
0619 20:44:40.256138 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:40.256138 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:40.256148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:40.256211 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:44:40.256353 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.256871 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.258044 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.258242 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.258298 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.258534 (+ 236us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:44:40.259323 (+ 789us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 20:44:40.259500 (+ 177us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46494: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 20:44:40.259604 19576 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:40.255803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:46815 (local address 127.0.0.1:46494)
0619 20:44:40.255982 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:40.255995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:40.256068 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:40.256227 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:40.256228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:40.256230 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:44:40.256290 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:40.256292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.256964 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.256965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:40.257954 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:40.257956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.258324 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.258325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:40.258387 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.258393 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:44:40.259430 (+ 1037us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 20:44:40.259521 (+ 91us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.195.126:46815: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
W20260619 20:44:40.259646 19556 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46494: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260619 20:44:40.260167 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19480
E20260619 20:44:40.270689 19577 webserver.cc:533] Webserver: sslize failed:
E20260619 20:44:40.270769 19577 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T20:44:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T20:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:40Z Disabled control of system clock
I20260619 20:44:40.304639 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42527
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:42527
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/testchainca.cert with env {}
W20260619 20:44:40.384128 19581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:40.384332 19581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:40.384356 19581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:40.385526 19581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:40.385566 19581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:40.385579 19581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:40.385592 19581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:40.385604 19581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:44:40.385617 19581 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:44:40.385628 19581 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 20:44:40.387108 19581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:42527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:40.387355 19581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:40.387571 19581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:40.392504 19587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:40.392534 19586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:40.392578 19589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:40.392827 19581 server_base.cc:1034] running on GCE node
I20260619 20:44:40.393069 19581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:40.393334 19581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:40.394519 19581 hybrid_clock.cc:648] HybridClock initialized: now 1781901880394486 us; error 38 us; skew 500 ppm
I20260619 20:44:40.395774 19581 webserver.cc:460] Webserver started at http://127.13.195.126:34605/ using document root <none> and password file <none>
I20260619 20:44:40.396014 19581 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:40.396063 19581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:40.396229 19581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:40.397264 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "8fb8b11c3f07443492e5a846a27cf6dd"
format_stamp: "Formatted at 2026-06-19 20:44:40 on dist-test-slave-n2td"
server_key: "82289069a8b6228025161337e66f5c98"
server_key_iv: "5f16a22b6a23569f6ff6f79263ef2bfd"
server_key_version: "encryptionkey@0"
I20260619 20:44:40.397641 19581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "8fb8b11c3f07443492e5a846a27cf6dd"
format_stamp: "Formatted at 2026-06-19 20:44:40 on dist-test-slave-n2td"
server_key: "82289069a8b6228025161337e66f5c98"
server_key_iv: "5f16a22b6a23569f6ff6f79263ef2bfd"
server_key_version: "encryptionkey@0"
I20260619 20:44:40.399039 19581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:40.399729 19596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:40.399927 19581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:40.400015 19581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "8fb8b11c3f07443492e5a846a27cf6dd"
format_stamp: "Formatted at 2026-06-19 20:44:40 on dist-test-slave-n2td"
server_key: "82289069a8b6228025161337e66f5c98"
server_key_iv: "5f16a22b6a23569f6ff6f79263ef2bfd"
server_key_version: "encryptionkey@0"
I20260619 20:44:40.400070 19581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:40.415321 19581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:40.415572 19581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:40.415674 19581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:40.419358 19581 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:42527
I20260619 20:44:40.419411 19648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:42527 every 8 connection(s)
I20260619 20:44:40.419723 19581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:40.420673 19649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:40.422968 19649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd: Bootstrap starting.
I20260619 20:44:40.423545 19649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:40.423832 19649 log.cc:824] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:40.424479 19649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd: No bootstrap required, opened a new log
I20260619 20:44:40.426049 19649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42527 } }
I20260619 20:44:40.426200 19649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:40.426247 19649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb8b11c3f07443492e5a846a27cf6dd, State: Initialized, Role: FOLLOWER
I20260619 20:44:40.426432 19649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42527 } }
I20260619 20:44:40.426568 19649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:40.426614 19649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:40.426667 19649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:40.427157 19649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42527 } }
I20260619 20:44:40.427271 19649 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb8b11c3f07443492e5a846a27cf6dd; no voters:
I20260619 20:44:40.427452 19649 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:40.427489 19652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:40.427675 19652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [term 1 LEADER]: Becoming Leader. State: Replica: 8fb8b11c3f07443492e5a846a27cf6dd, State: Running, Role: LEADER
I20260619 20:44:40.427704 19649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:40.427775 19652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42527 } }
I20260619 20:44:40.428191 19653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42527 } } }
I20260619 20:44:40.428272 19654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb8b11c3f07443492e5a846a27cf6dd. Latest consensus state: current_term: 1 leader_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb8b11c3f07443492e5a846a27cf6dd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42527 } } }
I20260619 20:44:40.428345 19654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:40.428285 19653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:40.428598 19661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:40.428781 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19581
I20260619 20:44:40.428898 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:40.429083 19661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:40.429111 14093 external_mini_cluster.cc:1428] Setting key a802ba43829c08aa0f3c391dcc4576b2
I20260619 20:44:40.431058 19661 catalog_manager.cc:1347] Generated new cluster ID: 31323dddfe164c06b2e5da33f764f892
I20260619 20:44:40.431108 19661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:40.434307 19671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:40.429632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.13.195.126:42527)
0619 20:44:40.429873 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:40.429880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:40.429901 (+ 21us) server_negotiation.cc:407] Connection header received
0619 20:44:40.430045 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:40.430050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:40.430135 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:40.430269 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:40.430518 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.431638 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.432431 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.432864 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.433003 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.433674 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:40.433677 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:40.433677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:44:40.433866 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:40.433897 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:44:40.433907 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:40.433982 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:40.434057 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:44:40.434136 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
I20260619 20:44:40.435196 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:44:40.444170 19661 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:40.444659 19661 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:40.449666 19661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb8b11c3f07443492e5a846a27cf6dd: Generated new TSK 0
I20260619 20:44:40.449852 19661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:44:40.514825 19671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:40.459400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.13.195.126:42527)
0619 20:44:40.459448 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:40.459450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:40.459555 (+ 105us) server_negotiation.cc:407] Connection header received
0619 20:44:40.459624 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:40.459624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:40.459634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:40.459692 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:44:40.459832 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.460308 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.461532 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.461709 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.461761 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.461957 (+ 196us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:44:40.514580 (+ 52623us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 20:44:40.514759 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49584: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 20:44:40.514916 19671 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49584: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 20:44:40.515018 19678 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:40.459319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42527 (local address 127.0.0.1:49584)
0619 20:44:40.459480 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:40.459491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:40.459555 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:40.459711 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:40.459712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:40.459714 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:44:40.459771 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:40.459773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.460386 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.460387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:40.461441 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:40.461445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:40.461788 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:40.461789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:40.461861 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:40.461868 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:44:40.514755 (+ 52887us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 20:44:40.514915 (+ 160us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.195.126:42527: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260619 20:44:40.515477 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19581
2026-06-19T20:44:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (235 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:40 dist-test-slave-n2td krb5kdc[19690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:40 dist-test-slave-n2td krb5kdc[19690](info): set up 2 sockets
Jun 19 20:44:40 dist-test-slave-n2td krb5kdc[19690](info): commencing operation
krb5kdc: starting...
W20260619 20:44:41.749542 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:41 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:41.867573 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:38339
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:38339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:41.949877 19703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:41.950071 19703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:41.950094 19703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:41.951313 19703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:41.951357 19703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:41.951372 19703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:41.951385 19703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:41.951397 19703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:41.952878 19703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:38339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:38339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:41.953120 19703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:41.953341 19703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:41.958011 19708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:41.958024 19711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:41.958024 19709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:41.958280 19703 server_base.cc:1034] running on GCE node
I20260619 20:44:41.958666 19703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:41.958961 19703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:41.960151 19703 hybrid_clock.cc:648] HybridClock initialized: now 1781901881960138 us; error 28 us; skew 500 ppm
Jun 19 20:44:41 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901881, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:41.962747 19703 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:41.963241 19703 webserver.cc:460] Webserver started at http://127.13.195.126:32895/ using document root <none> and password file <none>
I20260619 20:44:41.963490 19703 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:41.963564 19703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:41.963725 19703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:41.964898 19703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "8619ea89ec8a44b096e4b12b27c1e1b3"
format_stamp: "Formatted at 2026-06-19 20:44:41 on dist-test-slave-n2td"
server_key: "32f502fabf1ad202b23e8218e61b50f3"
server_key_iv: "e5f423d14cd8c8524cedfcd6ccf4552d"
server_key_version: "encryptionkey@0"
I20260619 20:44:41.965270 19703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "8619ea89ec8a44b096e4b12b27c1e1b3"
format_stamp: "Formatted at 2026-06-19 20:44:41 on dist-test-slave-n2td"
server_key: "32f502fabf1ad202b23e8218e61b50f3"
server_key_iv: "e5f423d14cd8c8524cedfcd6ccf4552d"
server_key_version: "encryptionkey@0"
I20260619 20:44:41.966840 19703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:41.967561 19718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:41.967734 19703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:41.967856 19703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "8619ea89ec8a44b096e4b12b27c1e1b3"
format_stamp: "Formatted at 2026-06-19 20:44:41 on dist-test-slave-n2td"
server_key: "32f502fabf1ad202b23e8218e61b50f3"
server_key_iv: "e5f423d14cd8c8524cedfcd6ccf4552d"
server_key_version: "encryptionkey@0"
I20260619 20:44:41.967936 19703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:41.997572 19703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:42.000952 19703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:42.001132 19703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:42.006671 19703 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:38339
I20260619 20:44:42.006701 19771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:38339 every 8 connection(s)
I20260619 20:44:42.007175 19703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:42.008304 19772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:42.011925 19772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3: Bootstrap starting.
I20260619 20:44:42.012770 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19703
I20260619 20:44:42.012890 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:42.013041 14093 external_mini_cluster.cc:1428] Setting key 18df28d09530f8289814a832cc317ad9
I20260619 20:44:42.013334 19772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:42.013720 19772 log.cc:824] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:42.014580 19772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3: No bootstrap required, opened a new log
I20260619 20:44:42.017508 19772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38339 } }
I20260619 20:44:42.017692 19772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:42.017737 19772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8619ea89ec8a44b096e4b12b27c1e1b3, State: Initialized, Role: FOLLOWER
I20260619 20:44:42.017882 19772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38339 } }
I20260619 20:44:42.017970 19772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:42.018010 19772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:42.018059 19772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:42.018944 19772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38339 } }
I20260619 20:44:42.019078 19772 leader_election.cc:302] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8619ea89ec8a44b096e4b12b27c1e1b3; no voters:
I20260619 20:44:42.019273 19772 leader_election.cc:288] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:42.019304 19777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:42.019447 19777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [term 1 LEADER]: Becoming Leader. State: Replica: 8619ea89ec8a44b096e4b12b27c1e1b3, State: Running, Role: LEADER
I20260619 20:44:42.019556 19772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:42.019538 19777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38339 } }
I20260619 20:44:42.020022 19778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38339 } } }
I20260619 20:44:42.020150 19778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:42.020020 19779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8619ea89ec8a44b096e4b12b27c1e1b3. Latest consensus state: current_term: 1 leader_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8619ea89ec8a44b096e4b12b27c1e1b3" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38339 } } }
I20260619 20:44:42.020283 19779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:42.020385 19786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:42.020880 19786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:44:42 dist-test-slave-n2td krb5kdc[19690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:42.027479 19786 catalog_manager.cc:1347] Generated new cluster ID: 3a9ac07450c044d996dfc0cb9d542707
I20260619 20:44:42.027536 19786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:42.038657 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:42.013728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.13.195.126:38339)
0619 20:44:42.013901 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:42.013913 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:42.014224 (+ 311us) server_negotiation.cc:407] Connection header received
0619 20:44:42.014842 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:42.014851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:42.014919 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:42.015096 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:42.015905 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:42.017196 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:42.018173 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:42.018953 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:42.019188 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:42.023058 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:42.023062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:42.023062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:42.023075 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:42.036552 (+ 13477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:42.036973 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:42.036976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:42.036976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:42.037091 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:42.037286 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:42.037287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:42.037287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:42.037590 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:42.037733 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:42.038244 (+ 511us) server_negotiation.cc:299] Negotiation successful
0619 20:44:42.038380 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260619 20:44:42.068357 19786 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:42.068974 19786 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:42.082353 19786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8619ea89ec8a44b096e4b12b27c1e1b3: Generated new TSK 0
I20260619 20:44:42.082604 19786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
W20260619 20:44:43.302704 14093 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.13.195.65: real 1.215s user 0.000s sys 0.004s
I20260619 20:44:43.303130 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:38339
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:43.385218 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:43.385416 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:43.385437 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:43.386659 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:43.386700 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:43.386725 19798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:43.388221 19798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:38339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:43.388473 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:43.388723 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:43.393915 19804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:43.393918 19803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:43.393918 19806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:43.393932 19798 server_base.cc:1034] running on GCE node
I20260619 20:44:43.394338 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:43.394629 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:43.395810 19798 hybrid_clock.cc:648] HybridClock initialized: now 1781901883395797 us; error 29 us; skew 500 ppm
Jun 19 20:44:43 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901883, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:43.401839 19798 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:43.402341 19798 webserver.cc:460] Webserver started at http://127.13.195.65:33897/ using document root <none> and password file <none>
I20260619 20:44:43.402617 19798 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:43.402693 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:43.402849 19798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:43.403905 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "6036d05611d3482c8270a73be8c0ab01"
format_stamp: "Formatted at 2026-06-19 20:44:43 on dist-test-slave-n2td"
server_key: "9a3e5bbc14f8db90744ba6c6c30e4df1"
server_key_iv: "f9b6d9d4e74dbaaed00b2e394e1c9145"
server_key_version: "encryptionkey@0"
I20260619 20:44:43.404254 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "6036d05611d3482c8270a73be8c0ab01"
format_stamp: "Formatted at 2026-06-19 20:44:43 on dist-test-slave-n2td"
server_key: "9a3e5bbc14f8db90744ba6c6c30e4df1"
server_key_iv: "f9b6d9d4e74dbaaed00b2e394e1c9145"
server_key_version: "encryptionkey@0"
I20260619 20:44:43.405642 19798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:43.406378 19813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:43.406598 19798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:43.406713 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "6036d05611d3482c8270a73be8c0ab01"
format_stamp: "Formatted at 2026-06-19 20:44:43 on dist-test-slave-n2td"
server_key: "9a3e5bbc14f8db90744ba6c6c30e4df1"
server_key_iv: "f9b6d9d4e74dbaaed00b2e394e1c9145"
server_key_version: "encryptionkey@0"
I20260619 20:44:43.406786 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:43.428979 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:43.431069 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:43.431228 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:43.431458 19798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:43.431785 19798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:43.431818 19798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:43.431860 19798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:43.431885 19798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:43.437693 19798 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:33709
I20260619 20:44:43.437800 19926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:33709 every 8 connection(s)
I20260619 20:44:43.438091 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:43.438619 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19798
I20260619 20:44:43.438726 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:43.438984 14093 external_mini_cluster.cc:1428] Setting key b01471963ed2f1ba5e618cece92467db
Jun 19 20:44:43 dist-test-slave-n2td krb5kdc[19690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901883, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
I20260619 20:44:43.448125 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:43.438906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:54667 (local address 127.13.195.126:38339)
0619 20:44:43.439218 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:43.439220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:43.439232 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:44:43.440218 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:43.440219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:43.440232 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:43.440298 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:43.440922 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:43.441425 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:43.442447 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:43.442658 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:43.442737 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:43.445100 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:43.445101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:43.445102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:43.445110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:43.446718 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:43.447128 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:43.447130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:43.447130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:43.447192 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:43.447447 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:43.447447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:43.447448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:43.447567 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:43.447637 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:43.447917 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 20:44:43.448011 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":73,"threads_started":1}
I20260619 20:44:43.448307 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:43.438842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:38339 (local address 127.13.195.65:54667)
0619 20:44:43.439013 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:43.439062 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:43.440112 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:43.440365 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:43.440369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:43.440636 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:43.440828 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:43.440831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:43.441526 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:43.441527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:43.442337 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:43.442339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:43.442741 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:43.442742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:43.442882 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:43.443402 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:43.443411 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:43.445001 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:43.446806 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:43.446813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:43.446815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:43.447039 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:43.447257 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:43.447259 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:43.447259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:43.447371 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:43.447640 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:43.447647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:43.447787 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 20:44:43.447897 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 20:44:43.448074 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260619 20:44:43.448801 19927 heartbeater.cc:347] Connected to a master server at 127.13.195.126:38339
I20260619 20:44:43.448884 19927 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:43.449033 19927 heartbeater.cc:511] Master 127.13.195.126:38339 requested a full tablet report, sending...
I20260619 20:44:43.449504 19736 ts_manager.cc:194] Registered new tserver with Master: 6036d05611d3482c8270a73be8c0ab01 (127.13.195.65:33709)
I20260619 20:44:43.450150 19736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:54667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
W20260619 20:44:43.921965 14093 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.13.195.66: real 0.482s user 0.003s sys 0.000s
I20260619 20:44:43.922290 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:38339
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:44.002390 19933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:44.002694 19933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:44.002728 19933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:44.004024 19933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:44.004063 19933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:44.004091 19933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:44.005620 19933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:38339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:44.005868 19933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:44.006109 19933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:44.011708 19939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:44.011718 19941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:44.011721 19938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:44.011772 19933 server_base.cc:1034] running on GCE node
I20260619 20:44:44.012117 19933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:44.012364 19933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:44.013557 19933 hybrid_clock.cc:648] HybridClock initialized: now 1781901884013544 us; error 28 us; skew 500 ppm
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.016192 19933 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:44.016693 19933 webserver.cc:460] Webserver started at http://127.13.195.66:37391/ using document root <none> and password file <none>
I20260619 20:44:44.016940 19933 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:44.017009 19933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:44.017155 19933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:44.018204 19933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "13fa92f703344d4eb889b6c6df498429"
format_stamp: "Formatted at 2026-06-19 20:44:44 on dist-test-slave-n2td"
server_key: "b4c28dd8cb5f0556a97764ca09a8f645"
server_key_iv: "8413fedb720b8b0b3ae87c393bd0ae2b"
server_key_version: "encryptionkey@0"
I20260619 20:44:44.018610 19933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "13fa92f703344d4eb889b6c6df498429"
format_stamp: "Formatted at 2026-06-19 20:44:44 on dist-test-slave-n2td"
server_key: "b4c28dd8cb5f0556a97764ca09a8f645"
server_key_iv: "8413fedb720b8b0b3ae87c393bd0ae2b"
server_key_version: "encryptionkey@0"
I20260619 20:44:44.020071 19933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:44.020902 19948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:44.021127 19933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:44.021234 19933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "13fa92f703344d4eb889b6c6df498429"
format_stamp: "Formatted at 2026-06-19 20:44:44 on dist-test-slave-n2td"
server_key: "b4c28dd8cb5f0556a97764ca09a8f645"
server_key_iv: "8413fedb720b8b0b3ae87c393bd0ae2b"
server_key_version: "encryptionkey@0"
I20260619 20:44:44.021306 19933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:44.036116 19933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:44.038580 19933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:44.038743 19933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:44.038980 19933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:44.039291 19933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:44.039353 19933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:44.039397 19933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:44.039492 19933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:44.045487 19933 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:44109
I20260619 20:44:44.045640 20061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:44109 every 8 connection(s)
I20260619 20:44:44.045958 19933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:44.046991 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 19933
I20260619 20:44:44.047084 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:44.047247 14093 external_mini_cluster.cc:1428] Setting key 9ee8a7f2e1752f7c835d4ee02382dc6f
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:44:44.057168 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:44.046735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:54071 (local address 127.13.195.126:38339)
0619 20:44:44.046889 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.046892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.047569 (+ 677us) server_negotiation.cc:407] Connection header received
0619 20:44:44.048798 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.048799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.048809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.048867 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.049734 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.050271 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.051362 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.051730 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.051809 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:44.054204 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:44.054205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:44.054206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:44.054216 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:44.055956 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.056314 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.056315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.056315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.056370 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.056622 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.056622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.056623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.056724 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:44.056774 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:44.057028 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 20:44:44.057097 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260619 20:44:44.057425 20064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:44.046729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:38339 (local address 127.13.195.66:54071)
0619 20:44:44.047421 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:44.047463 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:44.048687 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:44.048947 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:44.048951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:44.049339 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:44.049631 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:44.049636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.050401 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.050402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:44.051263 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:44.051265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.051820 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.051821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:44.051928 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:44.052486 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:44.052494 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:44.054108 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:44.056033 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:44.056038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:44.056040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:44.056244 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:44.056447 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:44.056448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:44.056449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:44.056558 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:44.056779 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:44.056783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:44.056891 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 20:44:44.057002 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 20:44:44.057167 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":69,"threads_started":1}
I20260619 20:44:44.057951 20062 heartbeater.cc:347] Connected to a master server at 127.13.195.126:38339
I20260619 20:44:44.058074 20062 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:44.058240 20062 heartbeater.cc:511] Master 127.13.195.126:38339 requested a full tablet report, sending...
I20260619 20:44:44.058612 19736 ts_manager.cc:194] Registered new tserver with Master: 13fa92f703344d4eb889b6c6df498429 (127.13.195.66:44109)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
I20260619 20:44:44.058990 19736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:54071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:44.073691 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:38339
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:44.155112 20068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:44.155313 20068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:44.155335 20068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:44.156397 20068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:44.156431 20068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:44.156457 20068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:44.158008 20068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:38339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:44.158185 20068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:44.158349 20068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:44.163122 20073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:44.163136 20074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:44.163247 20076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:44.163635 20068 server_base.cc:1034] running on GCE node
I20260619 20:44:44.163877 20068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:44.164119 20068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:44.165278 20068 hybrid_clock.cc:648] HybridClock initialized: now 1781901884165275 us; error 36 us; skew 500 ppm
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.167868 20068 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:44.168347 20068 webserver.cc:460] Webserver started at http://127.13.195.67:37827/ using document root <none> and password file <none>
I20260619 20:44:44.168579 20068 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:44.168648 20068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:44.168780 20068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:44.169824 20068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "28f62f8879804dca8bd585a532d0d37c"
format_stamp: "Formatted at 2026-06-19 20:44:44 on dist-test-slave-n2td"
server_key: "498deb90e1c53490a59126a7c9c5fce2"
server_key_iv: "1081a8cc47991009ed3b3d1fcb299ddb"
server_key_version: "encryptionkey@0"
I20260619 20:44:44.170207 20068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "28f62f8879804dca8bd585a532d0d37c"
format_stamp: "Formatted at 2026-06-19 20:44:44 on dist-test-slave-n2td"
server_key: "498deb90e1c53490a59126a7c9c5fce2"
server_key_iv: "1081a8cc47991009ed3b3d1fcb299ddb"
server_key_version: "encryptionkey@0"
I20260619 20:44:44.171727 20068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:44.172472 20083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:44.172644 20068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:44.172740 20068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "28f62f8879804dca8bd585a532d0d37c"
format_stamp: "Formatted at 2026-06-19 20:44:44 on dist-test-slave-n2td"
server_key: "498deb90e1c53490a59126a7c9c5fce2"
server_key_iv: "1081a8cc47991009ed3b3d1fcb299ddb"
server_key_version: "encryptionkey@0"
I20260619 20:44:44.172823 20068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:44.191011 20068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:44.192999 20068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:44.193169 20068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:44.193421 20068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:44.193739 20068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:44.193794 20068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:44.193837 20068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:44.193861 20068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:44.200037 20068 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:45927
I20260619 20:44:44.200075 20196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:45927 every 8 connection(s)
I20260619 20:44:44.200408 20068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:44.208513 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 20068
I20260619 20:44:44.208634 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:44.208839 14093 external_mini_cluster.cc:1428] Setting key 63a7c1bacbef1eba8fbb0c8de3efd6c8
I20260619 20:44:44.209594 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:44.201131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:41005 (local address 127.13.195.126:38339)
0619 20:44:44.201217 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.201220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.201394 (+ 174us) server_negotiation.cc:407] Connection header received
0619 20:44:44.202303 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.202304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.202313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.202361 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.203043 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.203536 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.204550 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.204738 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.204788 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:44.206882 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:44.206884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:44.206884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:44.206891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:44.208108 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.208482 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.208484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.208484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.208546 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.208828 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.208829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.208829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.208971 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:44.209083 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:44.209405 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 20:44:44.209503 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 20:44:44.209785 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:44.201096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:38339 (local address 127.13.195.67:41005)
0619 20:44:44.201271 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:44.201307 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:44.202215 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:44.202477 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:44.202482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:44.202754 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:44.202956 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:44.202960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.203614 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.203615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:44.204475 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:44.204477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.204804 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.204805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:44.204900 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:44.205442 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:44.205450 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:44.206801 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:44.208174 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:44.208179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:44.208181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:44.208388 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:44.208625 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:44.208626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:44.208627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:44.208760 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:44.209073 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:44.209080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:44.209242 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 20:44:44.209384 (+ 142us) client_negotiation.cc:240] Negotiation successful
0619 20:44:44.209543 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260619 20:44:44.210220 20197 heartbeater.cc:347] Connected to a master server at 127.13.195.126:38339
I20260619 20:44:44.210315 20197 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:44.210546 20197 heartbeater.cc:511] Master 127.13.195.126:38339 requested a full tablet report, sending...
I20260619 20:44:44.210932 19736 ts_manager.cc:194] Registered new tserver with Master: 28f62f8879804dca8bd585a532d0d37c (127.13.195.67:45927)
I20260619 20:44:44.211303 19736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:41005
I20260619 20:44:44.220538 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/test-admin.keytab.
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.235186 14093 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 20:44:44.236786 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:44.236164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.13.195.65:33709)
0619 20:44:44.236278 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.236292 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.236303 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:44:44.236329 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.236330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.236344 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.236407 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.236705 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50510: BlockingRecv error: recv got EOF from 127.0.0.1:50510 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
W20260619 20:44:44.236896 14093 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.237918 14093 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.238847 14093 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:44.246542 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:44.239420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.13.195.65:33709)
0619 20:44:44.239474 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.239476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.239487 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:44:44.239557 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.239558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.239569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.239643 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.240027 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.240714 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.241465 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.241789 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.241881 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:44.243940 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:44.243948 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:44.243949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:44.243956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:44.245322 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.245706 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.245710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.245710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.245767 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.245973 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.245974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.245974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.246110 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:44.246261 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:44.246334 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 20:44:44.246415 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 20:44:44.248412 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:44.247778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.13.195.65:33709)
0619 20:44:44.247834 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.247835 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.247911 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:44:44.247984 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.247985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.247995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.248049 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.248367 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50526: BlockingRecv error: recv got EOF from 127.0.0.1:50526 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 20:44:44.248521 14093 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.249423 14093 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.250262 14093 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:44.256932 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:44.250776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.13.195.65:33709)
0619 20:44:44.250827 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.250830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.250874 (+ 44us) server_negotiation.cc:407] Connection header received
0619 20:44:44.250950 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.250951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.250965 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.251026 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.251379 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.251821 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.252500 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.252686 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.252768 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:44.254746 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:44.254748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:44.254748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:44.254753 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:44.256073 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.256332 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.256332 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.256333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.256375 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.256532 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.256532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.256532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.256605 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:44.256679 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:44.256747 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 20:44:44.256800 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 20:44:44.258328 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:44.257851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.13.195.65:33709)
0619 20:44:44.257905 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.257907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.257941 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:44:44.257988 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.257988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.257995 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.258038 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.258268 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50538: BlockingRecv error: recv got EOF from 127.0.0.1:50538 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 20:44:44.258374 14093 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901857247738-14093-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.259269 14093 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:44.260061 14093 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[19690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:44:44.266090 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:44.260483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.13.195.65:33709)
0619 20:44:44.260558 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:44.260559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:44.260602 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:44:44.260638 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:44.260639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:44.260645 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:44.260688 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:44.260995 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.261413 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.262035 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:44.262200 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:44.262244 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:44.264116 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:44.264117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:44.264117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:44.264123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:44.265233 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.265510 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.265511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.265511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.265556 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:44.265692 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:44.265693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:44.265693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:44.265765 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:44.265841 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:44.265889 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 20:44:44.265949 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 20:44:44.266798 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19798
I20260619 20:44:44.272434 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19933
I20260619 20:44:44.276568 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 20068
I20260619 20:44:44.280725 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 19703
2026-06-19T20:44:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3591 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[20217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[20217](info): set up 2 sockets
Jun 19 20:44:44 dist-test-slave-n2td krb5kdc[20217](info): commencing operation
krb5kdc: starting...
W20260619 20:44:45.322170 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:45 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab.
Entry for principal oryx/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab.
I20260619 20:44:45.441533 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45621
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab
--principal=oryx/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:45.525341 20230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:45.525626 20230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:45.525655 20230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:45.526889 20230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:45.526930 20230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:45.526948 20230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:45.526960 20230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:45.526973 20230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:45.528478 20230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab
--principal=oryx/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:45.528723 20230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:45.528952 20230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:44:45.533877 20230 server_base.cc:1034] running on GCE node
W20260619 20:44:45.533851 20236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:45.533890 20238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:45.533841 20235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:45.534277 20230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:45.534633 20230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:45.535790 20230 hybrid_clock.cc:648] HybridClock initialized: now 1781901885535766 us; error 57 us; skew 500 ppm
Jun 19 20:44:45 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:45.538249 20230 init.cc:377] Logged in from keytab as oryx/127.13.195.126@KRBTEST.COM (short username oryx)
I20260619 20:44:45.538726 20230 webserver.cc:460] Webserver started at http://127.13.195.126:46057/ using document root <none> and password file <none>
I20260619 20:44:45.538969 20230 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:45.539036 20230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:45.539178 20230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:45.540189 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "add908bac93a43b1971ccd05c7ff3a3a"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "5b68cc72b8652ff559313fabff7617eb"
server_key_iv: "2af7beab59341af3526484675e3dd9a6"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.540542 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "add908bac93a43b1971ccd05c7ff3a3a"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "5b68cc72b8652ff559313fabff7617eb"
server_key_iv: "2af7beab59341af3526484675e3dd9a6"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.541839 20230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:45.542598 20245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.542773 20230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.542862 20230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "add908bac93a43b1971ccd05c7ff3a3a"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "5b68cc72b8652ff559313fabff7617eb"
server_key_iv: "2af7beab59341af3526484675e3dd9a6"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.542950 20230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:45.587494 20230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:45.589924 20230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:45.590093 20230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:45.594298 20230 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:45621
I20260619 20:44:45.594336 20297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:45621 every 8 connection(s)
I20260619 20:44:45.594739 20230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:45.595683 20298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:45.596611 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 20230
I20260619 20:44:45.596745 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:45.596947 14093 external_mini_cluster.cc:1428] Setting key 7142e658924f05df731b1581d55c3dc1
I20260619 20:44:45.598650 20298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: Bootstrap starting.
I20260619 20:44:45.599457 20298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:45.599792 20298 log.cc:824] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:45.602573 20298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: No bootstrap required, opened a new log
I20260619 20:44:45.605311 20298 raft_consensus.cc:348] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } }
I20260619 20:44:45.605512 20298 raft_consensus.cc:374] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:45.605548 20298 raft_consensus.cc:729] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add908bac93a43b1971ccd05c7ff3a3a, State: Initialized, Role: FOLLOWER
I20260619 20:44:45.605734 20298 consensus_queue.cc:260] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } }
I20260619 20:44:45.605839 20298 raft_consensus.cc:388] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:45.605872 20298 raft_consensus.cc:482] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Jun 19 20:44:45 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:45.605944 20298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:45.606828 20298 raft_consensus.cc:504] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } }
I20260619 20:44:45.606968 20298 leader_election.cc:302] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: add908bac93a43b1971ccd05c7ff3a3a; no voters:
I20260619 20:44:45.607148 20298 leader_election.cc:288] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:45.607344 20304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:45.607447 20298 sys_catalog.cc:564] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:45.607558 20304 raft_consensus.cc:686] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 1 LEADER]: Becoming Leader. State: Replica: add908bac93a43b1971ccd05c7ff3a3a, State: Running, Role: LEADER
I20260619 20:44:45.607661 20304 consensus_queue.cc:237] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } }
I20260619 20:44:45.607988 20304 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader add908bac93a43b1971ccd05c7ff3a3a. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } }
I20260619 20:44:45.608000 20303 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } }
I20260619 20:44:45.608096 20304 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:45.608103 20303 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:45.608292 20309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:45.608805 20309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:45.610162 20320 mvcc.cc:205] Tried to move back new op lower bound from 7298670123457425408 to 7298670123449315328. Current Snapshot: MvccSnapshot[applied={T|T < 7298670123457425408}]
I20260619 20:44:45.610551 20309 catalog_manager.cc:1347] Generated new cluster ID: 844152427beb4df6be97f5b808e7bf76
I20260619 20:44:45.610605 20309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:45.611251 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:45.597462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.13.195.126:45621)
0619 20:44:45.597661 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:45.597672 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:45.597704 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:44:45.598318 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:45.598327 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:45.598391 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:45.598591 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:45.599535 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.601390 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.602502 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.603260 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.603377 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:45.606661 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:45.606666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:45.606667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:45.606682 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:45.609333 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:45.609780 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:45.609783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:45.609783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:45.609881 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:45.610101 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:45.610102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:45.610103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:45.610483 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:45.610628 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:45.610989 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 20:44:45.611105 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260619 20:44:45.655541 20309 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:45.656117 20309 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:45.659968 20309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: Generated new TSK 0
I20260619 20:44:45.660161 20309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab.
Entry for principal oryx/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab.
I20260619 20:44:45.696843 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45621
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab
--principal=oryx/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:45.779543 20324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:45.779752 20324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:45.779776 20324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:45.780879 20324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:45.780917 20324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:45.780943 20324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:45.782493 20324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab
--principal=oryx/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:45.782754 20324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:45.782997 20324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:45.787961 20330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:45.788007 20324 server_base.cc:1034] running on GCE node
W20260619 20:44:45.787982 20329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:45.787973 20332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:45.788410 20324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:45.788609 20324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:45.789777 20324 hybrid_clock.cc:648] HybridClock initialized: now 1781901885789771 us; error 34 us; skew 500 ppm
Jun 19 20:44:45 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:45.792279 20324 init.cc:377] Logged in from keytab as oryx/127.13.195.65@KRBTEST.COM (short username oryx)
I20260619 20:44:45.792773 20324 webserver.cc:460] Webserver started at http://127.13.195.65:35437/ using document root <none> and password file <none>
I20260619 20:44:45.793012 20324 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:45.793085 20324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:45.793231 20324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:45.794276 20324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "875aa58d46f948a7ab7216a45fb133dc"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "2d253c3cb5b162028e5733f914e0b2c6"
server_key_iv: "92d4fc6d08ebfc3ad183203c67eaa435"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.794656 20324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "875aa58d46f948a7ab7216a45fb133dc"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "2d253c3cb5b162028e5733f914e0b2c6"
server_key_iv: "92d4fc6d08ebfc3ad183203c67eaa435"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.795950 20324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:45.796717 20339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.796906 20324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:45.797011 20324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "875aa58d46f948a7ab7216a45fb133dc"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "2d253c3cb5b162028e5733f914e0b2c6"
server_key_iv: "92d4fc6d08ebfc3ad183203c67eaa435"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.797096 20324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:45.809988 20324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:45.812240 20324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:45.812408 20324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:45.812640 20324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:45.812948 20324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:45.813040 20324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.813102 20324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:45.813133 20324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.818780 20324 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:41687
I20260619 20:44:45.818823 20452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:41687 every 8 connection(s)
I20260619 20:44:45.819142 20324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:45.821647 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 20324
I20260619 20:44:45.821789 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:45.821978 14093 external_mini_cluster.cc:1428] Setting key 070f16169f9b4828a47d19d33eca98ec
Jun 19 20:44:45 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:45.829313 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:45.819861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:46441 (local address 127.13.195.126:45621)
0619 20:44:45.819925 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:45.819927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:45.820123 (+ 196us) server_negotiation.cc:407] Connection header received
0619 20:44:45.820595 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:45.820596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:45.820607 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:45.820669 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:45.821608 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.822285 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.823790 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.823995 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.824098 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:45.826487 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:45.826488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:45.826489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:45.826503 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:45.827917 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:45.828338 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:45.828339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:45.828339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:45.828402 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:45.828662 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:45.828663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:45.828664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:45.828780 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:45.828857 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:45.829127 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 20:44:45.829221 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:44:45.829506 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:45.819835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:46441)
0619 20:44:45.819979 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:45.820021 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:45.820499 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:45.820725 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:45.820728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:45.821278 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:45.821511 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:45.821518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.822387 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.822388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:45.823695 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:45.823698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.824109 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.824110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:45.824271 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:45.824822 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:45.824831 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:45.826363 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:45.828015 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:45.828023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:45.828025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:45.828259 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:45.828470 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:45.828471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:45.828472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:45.828593 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:45.828848 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:45.828854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:45.828997 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:44:45.829106 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:44:45.829261 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260619 20:44:45.830000 20453 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45621
I20260619 20:44:45.830111 20453 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:45.830307 20453 heartbeater.cc:511] Master 127.13.195.126:45621 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.13.195.66@KRBTEST.COM; defaulting to no policy
I20260619 20:44:45.830884 20262 ts_manager.cc:194] Registered new tserver with Master: 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687)
I20260619 20:44:45.831439 20262 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:46441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab.
Entry for principal oryx/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab.
I20260619 20:44:45.848950 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45621
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab
--principal=oryx/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:45.928957 20458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:45.929203 20458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:45.929226 20458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:45.930485 20458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:45.930541 20458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:45.930591 20458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:45.932056 20458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab
--principal=oryx/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:45.932305 20458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:45.932546 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:45.937541 20466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:45.937574 20464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:45.937603 20463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:45.937750 20458 server_base.cc:1034] running on GCE node
I20260619 20:44:45.937929 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:45.938133 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:45.939268 20458 hybrid_clock.cc:648] HybridClock initialized: now 1781901885939255 us; error 30 us; skew 500 ppm
Jun 19 20:44:45 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:45.941601 20458 init.cc:377] Logged in from keytab as oryx/127.13.195.66@KRBTEST.COM (short username oryx)
I20260619 20:44:45.942071 20458 webserver.cc:460] Webserver started at http://127.13.195.66:38423/ using document root <none> and password file <none>
I20260619 20:44:45.942312 20458 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:45.942358 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:45.942556 20458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:45.943593 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "0776a7d676f649dcb7e7c469b3148bc3"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "669fd7fad8fe48b32d30d045401d5ef8"
server_key_iv: "91c5f2c5f93735ab9e34cd02c7e3eccf"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.943948 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "0776a7d676f649dcb7e7c469b3148bc3"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "669fd7fad8fe48b32d30d045401d5ef8"
server_key_iv: "91c5f2c5f93735ab9e34cd02c7e3eccf"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.945255 20458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:45.946023 20473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.946208 20458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:45.946313 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "0776a7d676f649dcb7e7c469b3148bc3"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "669fd7fad8fe48b32d30d045401d5ef8"
server_key_iv: "91c5f2c5f93735ab9e34cd02c7e3eccf"
server_key_version: "encryptionkey@0"
I20260619 20:44:45.946368 20458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:45.957283 20458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:45.959589 20458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:45.959703 20458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:45.959884 20458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:45.960202 20458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:45.960238 20458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.960284 20458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:45.960310 20458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:45.965998 20458 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:41405
I20260619 20:44:45.966079 20586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:41405 every 8 connection(s)
I20260619 20:44:45.966389 20458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:44:45 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:45.973573 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 20458
I20260619 20:44:45.973665 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:45.973891 14093 external_mini_cluster.cc:1428] Setting key 4cb5fdd0f2d46299071afa6f6a3774d2
I20260619 20:44:45.975558 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:45.967136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:56567 (local address 127.13.195.126:45621)
0619 20:44:45.967204 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:45.967207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:45.967407 (+ 200us) server_negotiation.cc:407] Connection header received
0619 20:44:45.967884 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:45.967885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:45.967893 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:45.967959 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:45.968851 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.969338 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.970311 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.970556 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.970609 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:45.972682 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:45.972683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:45.972683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:45.972689 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:45.974060 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:45.974561 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:45.974563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:45.974563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:45.974626 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:45.974902 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:45.974903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:45.974904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:45.975021 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:45.975099 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:45.975381 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 20:44:45.975461 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:44:45.975739 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:45.967123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:56567)
0619 20:44:45.967278 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:45.967320 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:45.967801 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:45.968018 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:45.968022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:45.968586 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:45.968774 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:45.968778 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.969418 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.969419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:45.970235 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:45.970237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:45.970626 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:45.970627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:45.970732 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:45.971277 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:45.971286 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:45.972600 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:45.974118 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:45.974124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:45.974127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:45.974426 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:45.974708 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:45.974709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:45.974709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:45.974824 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:45.975089 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:45.975095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:45.975238 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:44:45.975342 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 20:44:45.975496 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260619 20:44:45.976194 20587 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45621
I20260619 20:44:45.976303 20587 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:45.976502 20587 heartbeater.cc:511] Master 127.13.195.126:45621 requested a full tablet report, sending...
I20260619 20:44:45.976892 20262 ts_manager.cc:194] Registered new tserver with Master: 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66:41405)
I20260619 20:44:45.977409 20262 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:56567
WARNING: no policy specified for oryx/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab.
Entry for principal oryx/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab.
I20260619 20:44:45.997305 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45621
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab
--principal=oryx/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:46.077345 20592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:46.077543 20592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:46.077565 20592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:46.078660 20592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:46.078693 20592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:46.078718 20592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:46.080077 20592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab
--principal=oryx/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:46.080262 20592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:46.080438 20592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:46.085623 20598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.085582 20600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.085582 20597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.085811 20592 server_base.cc:1034] running on GCE node
I20260619 20:44:46.086025 20592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:46.086288 20592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:46.087477 20592 hybrid_clock.cc:648] HybridClock initialized: now 1781901886087451 us; error 36 us; skew 500 ppm
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:46.090277 20592 init.cc:377] Logged in from keytab as oryx/127.13.195.67@KRBTEST.COM (short username oryx)
I20260619 20:44:46.090760 20592 webserver.cc:460] Webserver started at http://127.13.195.67:35353/ using document root <none> and password file <none>
I20260619 20:44:46.090977 20592 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.091063 20592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.091208 20592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:46.092294 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "138ae6c139c8489c9974c758cbf693fa"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "d76c34123a8c20c1c6fefdfc79ae192f"
server_key_iv: "8352963d9bf3c2c2166980296c814a97"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.092620 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "138ae6c139c8489c9974c758cbf693fa"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "d76c34123a8c20c1c6fefdfc79ae192f"
server_key_iv: "8352963d9bf3c2c2166980296c814a97"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.093920 20592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:46.094728 20607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.094897 20592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:46.094973 20592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "138ae6c139c8489c9974c758cbf693fa"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "d76c34123a8c20c1c6fefdfc79ae192f"
server_key_iv: "8352963d9bf3c2c2166980296c814a97"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.095026 20592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.112735 20592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:46.114571 20592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:46.114683 20592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:46.114862 20592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:46.115193 20592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:46.115227 20592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.115250 20592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:46.115294 20592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.121064 20592 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:38759
I20260619 20:44:46.121107 20720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:38759 every 8 connection(s)
I20260619 20:44:46.121418 20592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:46.121829 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 20592
I20260619 20:44:46.121932 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:46.122150 14093 external_mini_cluster.cc:1428] Setting key fd461e3810a60aebecd4d7d653843305
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:46.130908 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.122162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:33463 (local address 127.13.195.126:45621)
0619 20:44:46.122234 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.122237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.122452 (+ 215us) server_negotiation.cc:407] Connection header received
0619 20:44:46.123071 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.123073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.123084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.123138 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.124089 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.124594 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.125605 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.125815 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.125890 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.128012 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.128014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.128014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.128023 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.129462 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.129876 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.129878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.129878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.129940 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.130192 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.130193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.130193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.130312 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.130386 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.130736 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.130815 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:44:46.131102 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.122143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:33463)
0619 20:44:46.122314 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.122356 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.122955 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.123208 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.123212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.123790 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.123984 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.123989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.124687 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.124688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.125504 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.125506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.125893 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.125894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.126030 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.126587 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.126596 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.127911 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.129578 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.129584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.129587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.129801 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.130013 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.130014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.130014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.130124 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.130377 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.130384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.130596 (+ 212us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.130715 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.130870 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260619 20:44:46.131582 20721 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45621
I20260619 20:44:46.131676 20721 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:46.131848 20721 heartbeater.cc:511] Master 127.13.195.126:45621 requested a full tablet report, sending...
I20260619 20:44:46.132217 20262 ts_manager.cc:194] Registered new tserver with Master: 138ae6c139c8489c9974c758cbf693fa (127.13.195.67:38759)
I20260619 20:44:46.132601 20262 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:33463
I20260619 20:44:46.133726 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.195.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.125.keytab.
Entry for principal oryx/127.13.195.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.125.keytab.
I20260619 20:44:46.160830 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:41663
--webserver_interface=127.13.195.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.125.keytab
--principal=oryx/127.13.195.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45621,127.13.195.125:41663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:46.241364 20726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:46.241580 20726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:46.241602 20726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:46.242816 20726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:46.242861 20726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:46.242874 20726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:46.242887 20726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:46.242900 20726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:46.244415 20726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45621,127.13.195.125:41663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.125.keytab
--principal=oryx/127.13.195.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:41663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.195.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
W20260619 20:44:46.244552 20726 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.195.126:45621,127.13.195.125:41663'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 20:44:46.244709 20726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:46.244967 20726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:46.249938 20731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.249945 20734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.250085 20726 server_base.cc:1034] running on GCE node
W20260619 20:44:46.250154 20732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.250411 20726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:46.250763 20726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:46.251914 20726 hybrid_clock.cc:648] HybridClock initialized: now 1781901886251902 us; error 26 us; skew 500 ppm
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:46.254492 20726 init.cc:377] Logged in from keytab as oryx/127.13.195.125@KRBTEST.COM (short username oryx)
I20260619 20:44:46.254935 20726 webserver.cc:460] Webserver started at http://127.13.195.125:39071/ using document root <none> and password file <none>
I20260619 20:44:46.255144 20726 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.255184 20726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.255275 20726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:46.256274 20726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/instance:
uuid: "18c9747abdc943c9961bf7f236b73562"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "0e090f8ba515976d2223747d18454e7d"
server_key_iv: "4b5930ab96f5debdb09bf6315ac0ef0a"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.256640 20726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal/instance:
uuid: "18c9747abdc943c9961bf7f236b73562"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "0e090f8ba515976d2223747d18454e7d"
server_key_iv: "4b5930ab96f5debdb09bf6315ac0ef0a"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.257899 20726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:46.258625 20741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.258783 20726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:46.258869 20726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
uuid: "18c9747abdc943c9961bf7f236b73562"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "0e090f8ba515976d2223747d18454e7d"
server_key_iv: "4b5930ab96f5debdb09bf6315ac0ef0a"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.258945 20726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.273643 20726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:46.275588 20726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:46.275744 20726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:46.279686 20726 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.125:41663
I20260619 20:44:46.279767 20793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.125:41663 every 8 connection(s)
I20260619 20:44:46.280110 20726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
I20260619 20:44:46.281054 20794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:46.282645 20794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:44:46.285303 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 20726
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.125@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.195.125@KRBTEST.COM
I20260619 20:44:46.292865 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.283374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.13.195.126:45621)
0619 20:44:46.283445 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.283447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.283699 (+ 252us) server_negotiation.cc:407] Connection header received
0619 20:44:46.284134 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.284135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.284145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.284198 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.284792 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.285302 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.286869 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.287171 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.287240 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.289897 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.289899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.289899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.289905 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.291368 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.291845 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.291846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.291847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.291912 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.292175 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.292177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.292177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.292341 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.292424 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.292686 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.292772 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:44:46.293130 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.283366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52004)
0619 20:44:46.283580 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.283617 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.284053 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.284266 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.284269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.284530 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.284721 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.284725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.285391 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.285392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.286767 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.286770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.287263 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.287264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.287369 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.287926 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.287936 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.289783 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.291460 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.291466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.291469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.291771 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.291981 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.291982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.291982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.292092 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.292412 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.292417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.292551 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.292663 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.292811 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":116,"threads_started":1}
I20260619 20:44:46.293785 20794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:44:46.295099 20798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.285815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.13.195.125:41663)
0619 20:44:46.285958 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.285963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.285990 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:44:46.286058 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.286060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.286076 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.286135 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.286865 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.287814 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.288838 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.289313 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.289380 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.291850 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.291857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.291857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.291863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.293793 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.294221 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.294224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.294224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.294277 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.294508 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.294509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.294509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.294640 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.294854 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.294909 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.294990 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.125@KRBTEST.COM for oryx/127.13.195.125@KRBTEST.COM
I20260619 20:44:46.300685 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.294044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.0.0.1:39996)
0619 20:44:46.294094 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.294104 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.294139 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.294351 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.294352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.294517 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.294544 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.294546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.295248 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.295248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.295851 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.295853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.296163 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.296164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.296214 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.296632 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.296636 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.297678 (+ 1042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.299867 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.299867 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.299868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.300024 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.300199 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.300200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.300200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.300247 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.300418 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.300419 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.300450 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.300498 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.300618 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:44:46.300856 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.294108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.13.195.125:41663)
0619 20:44:46.294241 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.294243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.294248 (+ 5us) server_negotiation.cc:407] Connection header received
0619 20:44:46.294263 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.294263 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.294271 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.294350 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.294618 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.295159 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.295916 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.296098 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.296166 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.297748 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.297749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.297749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.297755 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.299805 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.300085 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.300086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.300086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.300134 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.300284 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.300285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.300285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.300371 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.300414 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.300511 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.300682 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
I20260619 20:44:46.302119 20794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Bootstrap starting.
I20260619 20:44:46.302789 20794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:46.303064 20794 log.cc:824] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:46.303772 20794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: No bootstrap required, opened a new log
I20260619 20:44:46.305115 20794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } }
I20260619 20:44:46.305230 20794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:46.305253 20794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c9747abdc943c9961bf7f236b73562, State: Initialized, Role: FOLLOWER
I20260619 20:44:46.305401 20794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } }
I20260619 20:44:46.305637 20801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } } }
I20260619 20:44:46.305740 20801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:44:46.305680 20794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 20:44:46.307361 20812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:44:46.307418 20812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:44:46.308945 20726 master_runner.cc:428] Detected that this master 18c9747abdc943c9961bf7f236b73562 is joining an existing cluster
I20260619 20:44:46.308983 20726 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.195.125:41663
I20260619 20:44:46.316643 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.309882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52018)
0619 20:44:46.310037 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.310049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.310102 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.310350 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.310351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.310577 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.310667 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.310669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.311639 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.311641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.312547 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.312549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.313035 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.313036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.313099 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.313471 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.313476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.313869 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.315584 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.315586 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.315586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.315783 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.315982 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.315983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.315983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.316041 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.316304 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.316305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.316349 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.316432 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.316529 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260619 20:44:46.316689 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.309960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.13.195.126:45621)
0619 20:44:46.310229 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.310231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.310240 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:46.310264 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.310265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.310275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.310331 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.310754 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.311554 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.312658 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.312923 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.313025 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.313927 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.313928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.313929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.313936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.315483 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.315869 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.315870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.315871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.315928 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.316094 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.316094 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.316095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.316225 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.316305 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.316477 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.316576 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260619 20:44:46.317288 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.310125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.0.0.1:40000)
0619 20:44:46.310218 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.310228 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.310277 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.310567 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.310567 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.310754 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.310807 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.310808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.311645 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.311646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.312218 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.312219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.312607 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.312608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.312666 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.313175 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.313181 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.313786 (+ 605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.315634 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.315634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.315635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.315874 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.316048 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.316048 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.316049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.316678 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.316948 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.316948 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.316970 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.317048 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.317146 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":42,"thread_start_us":55,"threads_started":1}
I20260619 20:44:46.317518 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.310377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40000 (local address 127.13.195.125:41663)
0619 20:44:46.310424 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.310426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.310468 (+ 42us) server_negotiation.cc:407] Connection header received
0619 20:44:46.310484 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.310484 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.310492 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.310539 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.310885 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.311569 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.312298 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.312512 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.312596 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.313874 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.313875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.313875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.313882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.315549 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.315943 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.315944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.315944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.315992 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.316760 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.316761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.316761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.316885 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.316917 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.317068 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.317198 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":33}
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.126@KRBTEST.COM for oryx/127.13.195.125@KRBTEST.COM
I20260619 20:44:46.326330 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.319346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.0.0.1:40004)
0619 20:44:46.319459 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.319505 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.319571 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.319728 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.319731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.320003 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.320054 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.320056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.320926 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.320926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.321490 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.321501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.321942 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.321943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.321997 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.322479 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.322485 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.323764 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.325360 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.325366 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.325368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.325560 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.325752 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.325752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.325753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.325804 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.326058 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.326064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.326103 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.326167 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.326251 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260619 20:44:46.326383 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.319390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40004 (local address 127.13.195.125:41663)
0619 20:44:46.319427 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.319430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.319569 (+ 139us) server_negotiation.cc:407] Connection header received
0619 20:44:46.319638 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.319639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.319649 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.319705 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.320124 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.320847 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.321575 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.321870 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.321939 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.323843 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.323844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.323844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.323851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.325280 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.325629 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.325631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.325631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.325687 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.325862 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.325863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.325864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.325996 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.326063 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.326220 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.326294 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:44:46.326977 20262 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.195.125:41663: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "18c9747abdc943c9961bf7f236b73562"
member_type: NON_VOTER
last_known_addr {
host: "127.13.195.125"
port: 41663
}
attrs {
promote: true
}
}
dest_uuid: "add908bac93a43b1971ccd05c7ff3a3a"
cas_config_opid_index: -1
I20260619 20:44:46.327286 20262 consensus_queue.cc:237] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } }
I20260619 20:44:46.327769 20822 raft_consensus.cc:2932] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 18c9747abdc943c9961bf7f236b73562 (127.13.195.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } }
I20260619 20:44:46.328027 20768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:46.328845 20822 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.195.125:41663
I20260619 20:44:46.328928 20768 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Refusing update from remote peer add908bac93a43b1971ccd05c7ff3a3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 20:44:46.329002 20822 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } }
I20260619 20:44:46.329079 20822 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:46.329079 20726 master.cc:537] Master@127.13.195.125:41663 shutting down...
I20260619 20:44:46.329778 20821 consensus_queue.cc:1035] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:44:46.330775 20246 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 20:44:46.331310 20246 consensus_peers.cc:487] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a -> Peer 18c9747abdc943c9961bf7f236b73562 (127.13.195.125:41663): Couldn't send request to peer 18c9747abdc943c9961bf7f236b73562. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 20:44:46.331784 20726 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:46.331881 20726 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:46.331934 20726 tablet_replica.cc:330] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: stopping tablet replica
I20260619 20:44:46.334285 20726 master.cc:552] Master@127.13.195.125:41663 shutdown complete.
I20260619 20:44:46.334369 20726 master_runner.cc:305] Clearing existing system tablet
I20260619 20:44:46.334597 20726 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.334695 20726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.335522 20726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:46.335994 20826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.336123 20726 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.336177 20726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.336203 20726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
uuid: "18c9747abdc943c9961bf7f236b73562"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "0e090f8ba515976d2223747d18454e7d"
server_key_iv: "4b5930ab96f5debdb09bf6315ac0ef0a"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.336717 20726 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:46.338064 20726 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 20:44:46.338117 20726 log.cc:1196] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 20:44:46.338318 20726 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Deleting consensus metadata
I20260619 20:44:46.338441 20726 master_runner.cc:315] Copying system tablet from 127.13.195.126:45621
I20260619 20:44:46.339095 20726 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: tablet copy: Beginning tablet copy session from remote peer at address 127.13.195.126:45621
I20260619 20:44:46.345302 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.339408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.13.195.126:45621)
0619 20:44:46.339448 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.339450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.339513 (+ 63us) server_negotiation.cc:407] Connection header received
0619 20:44:46.339591 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.339591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.339602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.339659 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.339963 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.340731 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.341402 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.341708 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.341779 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.342763 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.342765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.342765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.342772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.344247 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.344549 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.344550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.344550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.344610 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.344775 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.344776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.344776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.344904 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.344979 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.345141 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.345215 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:44:46.345329 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.339352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52024)
0619 20:44:46.339461 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.339471 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.339532 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.339661 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.339662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.339843 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.339889 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.339891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.340808 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.340809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.341344 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.341345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.341782 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.341782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.341829 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.342299 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.342304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.342685 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.344324 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.344325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.344325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.344482 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.344674 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.344675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.344675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.344721 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.344965 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.344965 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.344999 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.345106 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.345206 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":72,"threads_started":1}
I20260619 20:44:46.345880 20282 tablet_copy_service.cc:140] P add908bac93a43b1971ccd05c7ff3a3a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 18c9747abdc943c9961bf7f236b73562 ({username='oryx', principal='oryx/127.13.195.125@KRBTEST.COM'} at 127.0.0.1:52024)
I20260619 20:44:46.345983 20282 tablet_copy_service.cc:161] P add908bac93a43b1971ccd05c7ff3a3a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 18c9747abdc943c9961bf7f236b73562 at {username='oryx', principal='oryx/127.13.195.125@KRBTEST.COM'} at 127.0.0.1:52024: session id = 18c9747abdc943c9961bf7f236b73562-00000000000000000000000000000000
I20260619 20:44:46.346782 20282 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 20:44:46.347357 20726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 20:44:46.348203 20726 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: tablet copy: Starting download of 0 data blocks...
I20260619 20:44:46.348351 20726 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: tablet copy: Starting download of 1 WAL segments...
I20260619 20:44:46.349181 20726 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 20:44:46.349982 20282 tablet_copy_service.cc:342] P add908bac93a43b1971ccd05c7ff3a3a: Request end of tablet copy session 18c9747abdc943c9961bf7f236b73562-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.195.125@KRBTEST.COM'} at 127.0.0.1:52024
I20260619 20:44:46.350044 20282 tablet_copy_service.cc:433] P add908bac93a43b1971ccd05c7ff3a3a: ending tablet copy session 18c9747abdc943c9961bf7f236b73562-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 18c9747abdc943c9961bf7f236b73562
I20260619 20:44:46.352905 20726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:46.354058 20833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.354137 20834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.354286 20836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.354491 20726 server_base.cc:1034] running on GCE node
I20260619 20:44:46.354588 20726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:46.354744 20726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:46.355871 20726 hybrid_clock.cc:648] HybridClock initialized: now 1781901886355865 us; error 26 us; skew 500 ppm
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:46.357121 20726 init.cc:377] Logged in from keytab as oryx/127.13.195.125@KRBTEST.COM (short username oryx)
I20260619 20:44:46.357390 20726 webserver.cc:460] Webserver started at http://127.13.195.125:36561/ using document root <none> and password file <none>
I20260619 20:44:46.357522 20726 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.357564 20726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.358148 20726 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.358727 20843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.358875 20726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.358947 20726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
uuid: "18c9747abdc943c9961bf7f236b73562"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "0e090f8ba515976d2223747d18454e7d"
server_key_iv: "4b5930ab96f5debdb09bf6315ac0ef0a"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.359018 20726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.382905 20726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:46.384840 20726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:46.387665 20726 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.125:41663
I20260619 20:44:46.387771 20895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.125:41663 every 8 connection(s)
I20260619 20:44:46.388002 20726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
I20260619 20:44:46.388933 20896 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:44:46.389472 20896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Bootstrap starting.
I20260619 20:44:46.392367 20896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:44:46.392617 20896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Bootstrap complete.
I20260619 20:44:46.392815 20896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } }
I20260619 20:44:46.392908 20896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 18c9747abdc943c9961bf7f236b73562, State: Initialized, Role: LEARNER
I20260619 20:44:46.392948 20896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } }
I20260619 20:44:46.393138 20896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:46.393167 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } }
I20260619 20:44:46.393285 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:44:46.394660 20910 catalog_manager.cc:1259] Loaded cluster ID: 844152427beb4df6be97f5b808e7bf76
I20260619 20:44:46.394708 20910 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: loading cluster ID for follower catalog manager: success
I20260619 20:44:46.395704 20910 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: acquiring CA information for follower catalog manager: success
I20260619 20:44:46.395973 20910 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:44:46.403280 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.396598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.13.195.125:41663)
0619 20:44:46.396715 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.396717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.396726 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:46.396751 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.396752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.396766 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.396839 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.397410 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.398091 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.398954 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.399160 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.399233 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.400579 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.400580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.400581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.400588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.402087 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.402466 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.402468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.402468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.402536 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.402738 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.402739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.402740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.402851 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.402956 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.403067 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.403150 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
WARNING: no policy specified for oryx/127.13.195.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.124.keytab.
Entry for principal oryx/127.13.195.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.124.keytab.
I20260619 20:44:46.429766 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:33719
--webserver_interface=127.13.195.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.124.keytab
--principal=oryx/127.13.195.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:46.517711 20914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:46.517928 20914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:46.517951 20914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:46.519093 20914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:46.519129 20914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:46.519142 20914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:46.519155 20914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:46.519167 20914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:46.520756 20914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.124.keytab
--principal=oryx/127.13.195.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:33719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.195.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:46.520995 20914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:46.521180 20914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:46.526748 20922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.526748 20919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.526780 20920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.527045 20914 server_base.cc:1034] running on GCE node
I20260619 20:44:46.527289 20914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:46.527575 20914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:46.528733 20914 hybrid_clock.cc:648] HybridClock initialized: now 1781901886528710 us; error 35 us; skew 500 ppm
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:46.531281 20914 init.cc:377] Logged in from keytab as oryx/127.13.195.124@KRBTEST.COM (short username oryx)
I20260619 20:44:46.531692 20914 webserver.cc:460] Webserver started at http://127.13.195.124:44293/ using document root <none> and password file <none>
I20260619 20:44:46.531929 20914 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.531996 20914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.532146 20914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:46.533174 20914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/instance:
uuid: "61e2db694ada433087397526ac6ce524"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "025dc253aa249d135db64ff0e4a7f2c5"
server_key_iv: "64d8ec8881f0ae71fafa3d269ec393af"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.533529 20914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal/instance:
uuid: "61e2db694ada433087397526ac6ce524"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "025dc253aa249d135db64ff0e4a7f2c5"
server_key_iv: "64d8ec8881f0ae71fafa3d269ec393af"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.534991 20914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:46.535758 20929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.535934 20914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:46.536041 20914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
uuid: "61e2db694ada433087397526ac6ce524"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "025dc253aa249d135db64ff0e4a7f2c5"
server_key_iv: "64d8ec8881f0ae71fafa3d269ec393af"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.536119 20914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.551086 20914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:46.553508 20914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:46.553644 20914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:46.561305 20914 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.124:33719
I20260619 20:44:46.561332 20981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.124:33719 every 8 connection(s)
I20260619 20:44:46.561733 20914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
I20260619 20:44:46.562810 20982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:46.564320 20982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:44:46.564985 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 20914
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.124@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.195.124@KRBTEST.COM
I20260619 20:44:46.574733 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.565232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.13.195.126:45621)
0619 20:44:46.565282 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.565284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.565453 (+ 169us) server_negotiation.cc:407] Connection header received
0619 20:44:46.566123 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.566125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.566138 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.566201 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.566850 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.567344 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.568353 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.568603 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.568653 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.571005 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.571007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.571007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.571015 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.572789 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.573303 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.573304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.573305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.573369 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.573884 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.573886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.573886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.574071 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.574156 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.574509 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.574612 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:44:46.574832 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.565097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52032)
0619 20:44:46.565275 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.565323 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.565998 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.566233 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.566236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.566575 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.566753 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.566759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.567438 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.567438 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.568271 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.568273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.568694 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.568695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.568803 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.569342 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.569350 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.570910 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.572905 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.572910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.572912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.573223 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.573562 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.573563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.573563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.573698 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.574147 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.574153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.574295 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.574375 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.574558 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260619 20:44:46.575462 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.565545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.13.195.124:33719)
0619 20:44:46.565694 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.565700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.565732 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:44:46.566001 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.566010 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.566038 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.566128 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.566786 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.567740 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.568471 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.568826 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.568893 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.571567 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.571573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.571574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.571581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.573404 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.573892 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.573895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.573896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.573964 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.574217 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.574218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.574218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.574415 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.575140 (+ 725us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.575201 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.575310 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260619 20:44:46.575464 20982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.124@KRBTEST.COM for oryx/127.13.195.125@KRBTEST.COM
I20260619 20:44:46.582070 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.575841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.0.0.1:40026)
0619 20:44:46.575997 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.576020 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.576087 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.576219 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.576220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.576344 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.576371 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.576372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.577003 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.577004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.577498 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.577500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.577870 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.577870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.577927 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.578331 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.578336 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.579529 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.581104 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.581105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.581105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.581267 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.581459 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.581460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.581460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.581514 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.581763 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.581764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.581804 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.581883 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.581970 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260619 20:44:46.582079 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.575941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40026 (local address 127.13.195.125:41663)
0619 20:44:46.575994 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.575996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.576073 (+ 77us) server_negotiation.cc:407] Connection header received
0619 20:44:46.576147 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.576148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.576161 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.576215 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.576446 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.576925 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.577582 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.577799 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.577873 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.579617 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.579618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.579619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.579626 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.581005 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.581337 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.581338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.581338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.581395 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.581585 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.581586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.581586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.581692 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.581760 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.581919 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.581995 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:44:46.582572 20982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.124@KRBTEST.COM for oryx/127.13.195.124@KRBTEST.COM
I20260619 20:44:46.588435 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.582827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.0.0.1:39574)
0619 20:44:46.582857 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.582866 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.582919 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.583040 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.583041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.583163 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.583184 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.583185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.583786 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.583787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.584241 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.584242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.584614 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.584615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.584659 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.585062 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.585067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.586129 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.587656 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.587656 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.587657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.587836 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.588003 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.588003 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.588003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.588052 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.588232 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.588233 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.588263 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.588312 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.588375 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 20:44:46.588538 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.582857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.13.195.124:33719)
0619 20:44:46.582903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.582905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.582913 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:46.582975 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.582976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.582985 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.583026 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.583259 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.583706 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.584326 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.584546 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.584617 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.586202 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.586203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.586203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.586209 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.587544 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.587900 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.587901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.587902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.587947 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.588102 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.588102 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.588102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.588185 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.588229 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.588325 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.588416 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":31}
I20260619 20:44:46.589711 20982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Bootstrap starting.
I20260619 20:44:46.590665 20982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:46.590891 20982 log.cc:824] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:46.591707 20982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: No bootstrap required, opened a new log
I20260619 20:44:46.594313 20982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } }
I20260619 20:44:46.594486 20982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:46.594542 20982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e2db694ada433087397526ac6ce524, State: Initialized, Role: FOLLOWER
I20260619 20:44:46.594697 20982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } }
I20260619 20:44:46.595034 20987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } } }
I20260619 20:44:46.595077 20982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:46.595141 20987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:44:46.597550 20998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:44:46.597610 20998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:44:46.599936 20914 master_runner.cc:428] Detected that this master 61e2db694ada433087397526ac6ce524 is joining an existing cluster
I20260619 20:44:46.599987 20914 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.195.124:33719
I20260619 20:44:46.619627 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.601621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39588 (local address 127.13.195.124:33719)
0619 20:44:46.601652 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.601653 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.601991 (+ 338us) server_negotiation.cc:407] Connection header received
0619 20:44:46.602089 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.602090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.602097 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.602134 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.602838 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.603326 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.604234 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.604426 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.604507 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.606033 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.606034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.606034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.606041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.607609 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.608078 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.608079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.608080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.618606 (+ 10526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.618945 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.618945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.618946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.619123 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.619170 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.619383 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.619491 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260619 20:44:46.619983 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.601734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.0.0.1:39588)
0619 20:44:46.601935 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.601945 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.602011 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.602222 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.602222 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.602610 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.602742 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.602743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.603415 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.603416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.604135 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.604137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.604559 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.604560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.604641 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.605521 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.605526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.605925 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.607698 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.607699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.607699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.607985 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.618693 (+ 10708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.618693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.618694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.618846 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.619217 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.619218 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.619268 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.619753 (+ 485us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.619855 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":107,"threads_started":1}
I20260619 20:44:46.622543 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.601332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52042)
0619 20:44:46.601863 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.601875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.601943 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.602169 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.602169 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.603566 (+ 1397us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.603626 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.603628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.606045 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.606046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.607202 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.607205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.607741 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.607741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.607844 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.608310 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.608314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.619097 (+ 10783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.621312 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.621313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.621314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.621532 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.621786 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.621787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.621788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.621849 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.622164 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.622165 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.622208 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.622268 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.622361 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":85,"threads_started":1}
I20260619 20:44:46.622635 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.601579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.13.195.126:45621)
0619 20:44:46.601800 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.601803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.601920 (+ 117us) server_negotiation.cc:407] Connection header received
0619 20:44:46.602020 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.602021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.602047 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.602100 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.604960 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.605935 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.607291 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.607627 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.607688 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.619216 (+ 11528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.619217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.619218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.619226 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.621200 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.621628 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.621629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.621629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.621711 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.621932 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.621933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.621933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.622081 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.622162 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.622315 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.622395 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260619 20:44:46.623698 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.601521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.13.195.125:41663)
0619 20:44:46.601628 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.601630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.601638 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:46.601659 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.601659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.601670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.601742 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.602395 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.603320 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.604917 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.605244 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.605381 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.619872 (+ 14491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.619873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.619873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.619881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.621724 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.622080 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.622080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.622081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.622140 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.622333 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.622334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.622334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.623254 (+ 920us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.623304 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.623506 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.623596 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 20:44:46.623922 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.601320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.0.0.1:40038)
0619 20:44:46.601532 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.601543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.601590 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.601752 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.601753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.602094 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.602318 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.602321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.603703 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.603704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.604832 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.604835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.605383 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.605384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.605511 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.618962 (+ 13451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.618967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.619766 (+ 799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.621814 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.621815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.621815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.622020 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.622203 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.622204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.622204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.622257 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.623356 (+ 1099us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.623356 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.623396 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.623473 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.623601 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":7,"thread_start_us":76,"threads_started":1}
I20260619 20:44:46.626845 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 20324
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901885, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.126@KRBTEST.COM for oryx/127.13.195.124@KRBTEST.COM
I20260619 20:44:46.632185 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.625188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.13.195.124:33719)
0619 20:44:46.625244 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.625246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.625254 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:46.625283 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.625283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.625292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.625340 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.625595 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.626038 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.627429 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.627625 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.627702 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.629418 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.629419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.629420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.629426 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.631047 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.631443 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.631444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.631444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.631508 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.631702 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.631703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.631703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.631804 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.631875 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.632024 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.632086 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:44:46.632190 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.625124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.0.0.1:39598)
0619 20:44:46.625171 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.625181 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.625218 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.625341 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.625342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.625485 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.625524 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.625525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.626114 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.626116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.626943 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.626945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.627699 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.627700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.627766 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.628183 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.628188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.629334 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.631153 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.631154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.631155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.631365 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.631580 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.631581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.631581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.631642 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.631867 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.631868 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.631909 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.631985 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.632081 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:44:46.632737 20262 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.195.124:33719: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "61e2db694ada433087397526ac6ce524"
member_type: NON_VOTER
last_known_addr {
host: "127.13.195.124"
port: 33719
}
attrs {
promote: true
}
}
dest_uuid: "add908bac93a43b1971ccd05c7ff3a3a"
cas_config_opid_index: 5
I20260619 20:44:46.633008 20262 consensus_queue.cc:237] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } }
I20260619 20:44:46.633550 20956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:46.633509 20822 raft_consensus.cc:2932] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 61e2db694ada433087397526ac6ce524 (127.13.195.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.634549 20822 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.195.124:33719
I20260619 20:44:46.634575 20956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Refusing update from remote peer add908bac93a43b1971ccd05c7ff3a3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 20:44:46.634606 20822 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.634660 20822 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:46.634763 20914 master.cc:537] Master@127.13.195.124:33719 shutting down...
I20260619 20:44:46.634830 20821 consensus_queue.cc:1035] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:44:46.635494 20247 consensus_peers.cc:487] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a -> Peer 61e2db694ada433087397526ac6ce524 (127.13.195.124:33719): Couldn't send request to peer 61e2db694ada433087397526ac6ce524. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 20:44:46.637573 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:41687
--local_ip_for_outbound_sockets=127.13.195.65
--tserver_master_addrs=127.13.195.126:45621
--webserver_port=35437
--webserver_interface=127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab
--principal=oryx/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:44:46.639125 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.633434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.0.0.1:40052)
0619 20:44:46.635383 (+ 1949us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.635394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.635430 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.635865 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.635866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.635868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:46.635906 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.635907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.636836 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.636837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.638019 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.638022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.638904 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.638905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.638972 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.638996 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.639028 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.639062 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1932}
I20260619 20:44:46.639623 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.633521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.13.195.125:41663)
0619 20:44:46.635719 (+ 2198us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.635720 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.635728 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:46.635745 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.635746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.635761 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.635850 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:46.635981 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.636743 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.638114 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.638808 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.639380 (+ 572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.639456 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.639478 (+ 22us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.639525 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2170}
I20260619 20:44:46.639626 20914 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:46.639724 20914 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:46.639768 20914 tablet_replica.cc:330] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: stopping tablet replica
I20260619 20:44:46.639816 20870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 LEARNER]: Refusing update from remote peer add908bac93a43b1971ccd05c7ff3a3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 20:44:46.640000 20821 consensus_queue.cc:1035] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 20:44:46.640470 20870 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 61e2db694ada433087397526ac6ce524 (127.13.195.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.641544 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader add908bac93a43b1971ccd05c7ff3a3a. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.641623 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:44:46.641654 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: NON_VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: true } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.641697 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:44:46.642591 20821 raft_consensus.cc:1050] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: attempting to promote NON_VOTER 18c9747abdc943c9961bf7f236b73562 to VOTER
I20260619 20:44:46.642814 20821 consensus_queue.cc:237] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } }
I20260619 20:44:46.643307 20870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 LEARNER]: Refusing update from remote peer add908bac93a43b1971ccd05c7ff3a3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 20:44:46.643496 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.643314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.0.0.1:39612)
0619 20:44:46.643410 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.643472 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260619 20:44:46.643844 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.643721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.0.0.1:39628)
0619 20:44:46.643778 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.643811 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:33719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:44:46.643863 21010 consensus_queue.cc:1035] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260619 20:44:46.644028 20247 consensus_peers.cc:487] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a -> Peer 61e2db694ada433087397526ac6ce524 (127.13.195.124:33719): Couldn't send request to peer 61e2db694ada433087397526ac6ce524. Status: Network error: Client connection negotiation failed: client connection to 127.13.195.124:33719: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 20:44:46.644639 20822 raft_consensus.cc:2932] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 18c9747abdc943c9961bf7f236b73562 (127.13.195.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.645465 20822 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.645550 20822 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:46.645612 20822 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.645663 20822 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:46.645861 20870 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 18c9747abdc943c9961bf7f236b73562 (127.13.195.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.646764 21009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.646919 21009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:44:46.653138 20914 master.cc:552] Master@127.13.195.124:33719 shutdown complete.
I20260619 20:44:46.653189 20914 master_runner.cc:305] Clearing existing system tablet
I20260619 20:44:46.653319 20914 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.653364 20914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.654079 20914 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:46.654645 21012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.654809 20914 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.654867 20914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.654891 20914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
uuid: "61e2db694ada433087397526ac6ce524"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "025dc253aa249d135db64ff0e4a7f2c5"
server_key_iv: "64d8ec8881f0ae71fafa3d269ec393af"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.655673 20914 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:46.657369 20914 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 20:44:46.657446 20914 log.cc:1196] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 20:44:46.657645 20914 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Deleting consensus metadata
I20260619 20:44:46.657765 20914 master_runner.cc:315] Copying system tablet from 127.13.195.126:45621
I20260619 20:44:46.658390 20914 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: tablet copy: Beginning tablet copy session from remote peer at address 127.13.195.126:45621
I20260619 20:44:46.669365 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.658865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.13.195.126:45621)
0619 20:44:46.658907 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.658910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.662630 (+ 3720us) server_negotiation.cc:407] Connection header received
0619 20:44:46.662696 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.662697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.662711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.662782 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.663171 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.663918 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.665006 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.665302 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.665373 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.666545 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.666546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.666547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.666554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.668194 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.668583 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.668584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.668584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.668647 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.668848 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.668849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.668849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.668981 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.669045 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.669202 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.669277 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:46.669396 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.658817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52056)
0619 20:44:46.662527 (+ 3710us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.662541 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.662624 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.662798 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.662799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.663016 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.663090 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.663092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.664018 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.664019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.664904 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.664907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.665384 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.665385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.665454 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.666007 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.666013 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.666394 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.668280 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.668281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.668282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.668505 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.668718 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.668719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.668719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.668783 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.669050 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.669051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.669096 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.669166 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.669264 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3686,"thread_start_us":62,"threads_started":1}
I20260619 20:44:46.670018 20282 tablet_copy_service.cc:140] P add908bac93a43b1971ccd05c7ff3a3a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 61e2db694ada433087397526ac6ce524 ({username='oryx', principal='oryx/127.13.195.124@KRBTEST.COM'} at 127.0.0.1:52056)
I20260619 20:44:46.670078 20282 tablet_copy_service.cc:161] P add908bac93a43b1971ccd05c7ff3a3a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 61e2db694ada433087397526ac6ce524 at {username='oryx', principal='oryx/127.13.195.124@KRBTEST.COM'} at 127.0.0.1:52056: session id = 61e2db694ada433087397526ac6ce524-00000000000000000000000000000000
I20260619 20:44:46.670663 20282 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 20:44:46.671160 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 20:44:46.672606 20914 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: tablet copy: Starting download of 0 data blocks...
I20260619 20:44:46.672751 20914 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: tablet copy: Starting download of 1 WAL segments...
I20260619 20:44:46.673435 20914 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 20:44:46.674271 20282 tablet_copy_service.cc:342] P add908bac93a43b1971ccd05c7ff3a3a: Request end of tablet copy session 61e2db694ada433087397526ac6ce524-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.195.124@KRBTEST.COM'} at 127.0.0.1:52056
I20260619 20:44:46.674333 20282 tablet_copy_service.cc:433] P add908bac93a43b1971ccd05c7ff3a3a: ending tablet copy session 61e2db694ada433087397526ac6ce524-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 61e2db694ada433087397526ac6ce524
I20260619 20:44:46.677047 20914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:46.678175 21019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.678226 21022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.678653 20914 server_base.cc:1034] running on GCE node
W20260619 20:44:46.678853 21020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.679055 20914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:46.679194 20914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:46.680315 20914 hybrid_clock.cc:648] HybridClock initialized: now 1781901886680306 us; error 25 us; skew 500 ppm
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:46.681783 20914 init.cc:377] Logged in from keytab as oryx/127.13.195.124@KRBTEST.COM (short username oryx)
I20260619 20:44:46.682039 20914 webserver.cc:460] Webserver started at http://127.13.195.124:45179/ using document root <none> and password file <none>
I20260619 20:44:46.682142 20914 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.682168 20914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.682898 20914 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:46.683467 21029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.683616 20914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.683665 20914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
uuid: "61e2db694ada433087397526ac6ce524"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "025dc253aa249d135db64ff0e4a7f2c5"
server_key_iv: "64d8ec8881f0ae71fafa3d269ec393af"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.683712 20914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.696591 20914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:46.698522 20914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:46.701416 20914 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.124:33719
I20260619 20:44:46.701725 21081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.124:33719 every 8 connection(s)
I20260619 20:44:46.701797 20914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
I20260619 20:44:46.702778 21082 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:44:46.703336 21082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Bootstrap starting.
I20260619 20:44:46.706246 21082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:44:46.706518 21082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: Bootstrap complete.
I20260619 20:44:46.706714 21082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } }
I20260619 20:44:46.706804 21082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 61e2db694ada433087397526ac6ce524, State: Initialized, Role: LEARNER
I20260619 20:44:46.706840 21082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } }
I20260619 20:44:46.707020 21082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:46.708334 21085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:46.708439 21085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:44:46.708689 21096 catalog_manager.cc:1259] Loaded cluster ID: 844152427beb4df6be97f5b808e7bf76
I20260619 20:44:46.708778 21096 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: loading cluster ID for follower catalog manager: success
I20260619 20:44:46.709951 21096 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: acquiring CA information for follower catalog manager: success
I20260619 20:44:46.710249 21096 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 20:44:46.742022 21007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:46.742268 21007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:46.742312 21007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:46.743590 21007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:46.743664 21007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:46.743712 21007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:46.745607 21007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab
--principal=oryx/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:41687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=35437
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:46.745904 21007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:46.746137 21007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:46.751813 21102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.752019 21101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.751823 21104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.752318 21007 server_base.cc:1034] running on GCE node
I20260619 20:44:46.752538 21007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:46.752835 21007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:46.754025 21007 hybrid_clock.cc:648] HybridClock initialized: now 1781901886753964 us; error 71 us; skew 500 ppm
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:46.757174 21007 init.cc:377] Logged in from keytab as oryx/127.13.195.65@KRBTEST.COM (short username oryx)
I20260619 20:44:46.757663 21007 webserver.cc:460] Webserver started at http://127.13.195.65:35437/ using document root <none> and password file <none>
I20260619 20:44:46.757907 21007 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.757990 21007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.759855 21007 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:46.760607 21111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.760907 21007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:46.761011 21007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "875aa58d46f948a7ab7216a45fb133dc"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "2d253c3cb5b162028e5733f914e0b2c6"
server_key_iv: "92d4fc6d08ebfc3ad183203c67eaa435"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.761445 21007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.774245 21007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:46.776643 21007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:46.776762 21007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:46.776944 21007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:46.777263 21007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:46.777302 21007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.777325 21007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:46.777339 21007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.788926 21007 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:41687
I20260619 20:44:46.789060 21224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:41687 every 8 connection(s)
I20260619 20:44:46.789408 21007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:46.793291 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 21007
I20260619 20:44:46.793501 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 20458
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.124@KRBTEST.COM
I20260619 20:44:46.801363 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:41405
--local_ip_for_outbound_sockets=127.13.195.66
--tserver_master_addrs=127.13.195.126:45621
--webserver_port=38423
--webserver_interface=127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab
--principal=oryx/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.125@KRBTEST.COM
I20260619 20:44:46.804008 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.790317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:49643 (local address 127.13.195.124:33719)
0619 20:44:46.790606 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.790608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.791203 (+ 595us) server_negotiation.cc:407] Connection header received
0619 20:44:46.792345 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.792346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.792360 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.792448 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.793756 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.794869 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.796562 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.796944 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.796989 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.800233 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.800234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.800234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.800242 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.802122 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.802793 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.802794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.802795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.802866 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.803203 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.803204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.803205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.803349 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.803402 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.803775 (+ 373us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.803872 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":132,"threads_started":1}
I20260619 20:44:46.804265 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.790489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:52267 (local address 127.13.195.126:45621)
0619 20:44:46.790538 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.790541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.791207 (+ 666us) server_negotiation.cc:407] Connection header received
0619 20:44:46.792326 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.792327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.792339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.792451 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.794261 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.795039 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.796203 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.796415 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.796500 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.800255 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.800257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.800257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.800265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.802898 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.803344 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.803345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.803346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.803408 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.803672 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.803673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.803673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.803801 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.803841 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.804075 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.804173 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:44:46.804602 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.790182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.13.195.65:49643)
0619 20:44:46.791048 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.791096 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.792220 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.792528 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.792529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.793192 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.793661 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.793666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.795085 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.795086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.796463 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.796466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.797069 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.797070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.797216 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.797748 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.797753 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.800144 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.802220 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.802228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.802230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.802694 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.802969 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.802970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.802970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.803126 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.803437 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.803443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.803614 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.804104 (+ 490us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.804281 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":100,"thread_start_us":467,"threads_started":1}
I20260619 20:44:46.804658 21231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.790182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:52267)
0619 20:44:46.791049 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.791095 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.792218 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.792525 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.792529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.793191 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.794160 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.794163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.795118 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.795119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.796096 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.796098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.796508 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.796510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.796678 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.797271 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.797280 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.799810 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.802999 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.802999 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.803000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.803256 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.803528 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.803529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.803529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.803593 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.803888 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.803889 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.803931 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.804377 (+ 446us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.804500 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":44,"thread_start_us":506,"threads_started":1}
I20260619 20:44:46.805225 21226 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45621
I20260619 20:44:46.805335 21226 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:46.805544 21225 heartbeater.cc:347] Connected to a master server at 127.13.195.124:33719
I20260619 20:44:46.805570 21226 heartbeater.cc:511] Master 127.13.195.126:45621 requested a full tablet report, sending...
I20260619 20:44:46.805588 21225 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:46.805639 21225 heartbeater.cc:511] Master 127.13.195.124:33719 requested a full tablet report, sending...
I20260619 20:44:46.805994 20262 ts_manager.cc:194] Re-registered known tserver with Master: 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687)
I20260619 20:44:46.806046 21037 ts_manager.cc:194] Registered new tserver with Master: 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687)
I20260619 20:44:46.806591 20262 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:52267
I20260619 20:44:46.809105 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.790834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:58563 (local address 127.13.195.125:41663)
0619 20:44:46.790874 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.790877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.791203 (+ 326us) server_negotiation.cc:407] Connection header received
0619 20:44:46.792352 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.792354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.792365 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.792413 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.799185 (+ 6772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.799997 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.801181 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.801522 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.801568 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.806236 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.806238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.806238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.806247 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.807961 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.808319 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.808320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.808320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.808375 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.808565 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.808566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.808566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.808711 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.808774 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.808921 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.809003 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:44:46.809510 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.790768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.13.195.65:58563)
0619 20:44:46.791143 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.791155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.792225 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.792568 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.792569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.793192 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.799091 (+ 5899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.799093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.800073 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.800074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.801069 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.801072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.801913 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.801914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.802028 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.802592 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.802598 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.806094 (+ 3496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.808053 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.808069 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.808070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.808234 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.808454 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.808454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.808455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.808515 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.808767 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.808767 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.808807 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.808898 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.809004 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":112,"thread_start_us":120,"threads_started":1}
I20260619 20:44:46.810050 21227 heartbeater.cc:347] Connected to a master server at 127.13.195.125:41663
I20260619 20:44:46.810112 21227 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:46.810189 21227 heartbeater.cc:511] Master 127.13.195.125:41663 requested a full tablet report, sending...
I20260619 20:44:46.810642 20860 ts_manager.cc:194] Registered new tserver with Master: 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687)
W20260619 20:44:46.887382 21233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:46.887584 21233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:46.887609 21233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:46.888871 21233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:46.888924 21233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:46.888959 21233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:46.890497 21233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab
--principal=oryx/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:41405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=38423
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:46.890681 21233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:46.890861 21233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:44:46.896898 21233 server_base.cc:1034] running on GCE node
W20260619 20:44:46.896759 21238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.896762 21239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:46.896768 21241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:46.897275 21233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:46.897538 21233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:46.898696 21233 hybrid_clock.cc:648] HybridClock initialized: now 1781901886898686 us; error 47 us; skew 500 ppm
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:46.901225 21233 init.cc:377] Logged in from keytab as oryx/127.13.195.66@KRBTEST.COM (short username oryx)
I20260619 20:44:46.901654 21233 webserver.cc:460] Webserver started at http://127.13.195.66:38423/ using document root <none> and password file <none>
I20260619 20:44:46.901863 21233 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:46.901921 21233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:46.903414 21233 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:46.904043 21248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.904247 21233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:46.904386 21233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "0776a7d676f649dcb7e7c469b3148bc3"
format_stamp: "Formatted at 2026-06-19 20:44:45 on dist-test-slave-n2td"
server_key: "669fd7fad8fe48b32d30d045401d5ef8"
server_key_iv: "91c5f2c5f93735ab9e34cd02c7e3eccf"
server_key_version: "encryptionkey@0"
I20260619 20:44:46.904873 21233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:46.927062 21233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:46.929796 21233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:46.929983 21233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:46.930267 21233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:46.930642 21233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:46.930706 21233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.930752 21233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:46.930804 21233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:46.937059 21233 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:41405
I20260619 20:44:46.937134 21361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:41405 every 8 connection(s)
I20260619 20:44:46.937551 21233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.125@KRBTEST.COM
Jun 19 20:44:46 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901886, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.124@KRBTEST.COM
I20260619 20:44:46.947604 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 21233
I20260619 20:44:46.947767 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 20592
I20260619 20:44:46.949700 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.938529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:54243 (local address 127.13.195.126:45621)
0619 20:44:46.938709 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.938712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.938814 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:44:46.939933 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.939935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.939949 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.939986 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.940934 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.941787 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.943045 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.943265 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.943300 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.946014 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.946015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.946016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.946024 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.947988 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.948486 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.948487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.948487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.948559 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.948842 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.948842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.948843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.949010 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.949057 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.949448 (+ 391us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.949553 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260619 20:44:46.950348 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.938335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:54243)
0619 20:44:46.938660 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.938710 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.939802 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.940061 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.940062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.940562 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.940814 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.940820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.941886 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.941886 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.942929 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.942932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.943375 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.943375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.943475 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.944230 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.944234 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.945910 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.948095 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.948100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.948103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.948398 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.948632 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.948632 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.948633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.948768 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.949105 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.949111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.949291 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.949820 (+ 529us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.950008 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":61,"thread_start_us":226,"threads_started":1}
I20260619 20:44:46.950485 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.938466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:38549 (local address 127.13.195.124:33719)
0619 20:44:46.938563 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.938566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.939053 (+ 487us) server_negotiation.cc:407] Connection header received
0619 20:44:46.939933 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.939935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.939949 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.939986 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.940933 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.941775 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.943046 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.943396 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.943468 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.947086 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.947088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.947088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.947096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.949084 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.949499 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.949500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.949500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.949567 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.949769 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.949770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.949770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.949930 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.949989 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.950191 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.950301 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:44:46.950798 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.938335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.13.195.66:38549)
0619 20:44:46.938988 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.938996 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.939816 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.940075 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.940076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.940589 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.940814 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.940820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.941886 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.941887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.942934 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.942935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.943474 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.943475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.943582 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.943990 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.943994 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.946971 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.949179 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.949180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.949181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.949417 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.949636 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.949637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.949637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.949699 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.950025 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.950026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.950071 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.950567 (+ 496us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.950679 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":343,"thread_start_us":85,"threads_started":1}
I20260619 20:44:46.950852 21363 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45621
I20260619 20:44:46.950945 21363 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:46.951134 21363 heartbeater.cc:511] Master 127.13.195.126:45621 requested a full tablet report, sending...
I20260619 20:44:46.951668 21362 heartbeater.cc:347] Connected to a master server at 127.13.195.124:33719
I20260619 20:44:46.951719 21362 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:46.951783 21362 heartbeater.cc:511] Master 127.13.195.124:33719 requested a full tablet report, sending...
I20260619 20:44:46.951799 20262 ts_manager.cc:194] Re-registered known tserver with Master: 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66:41405)
I20260619 20:44:46.952126 21037 ts_manager.cc:194] Registered new tserver with Master: 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66:41405)
I20260619 20:44:46.952790 20262 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:54243
I20260619 20:44:46.954908 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:38759
--local_ip_for_outbound_sockets=127.13.195.67
--tserver_master_addrs=127.13.195.126:45621
--webserver_port=35353
--webserver_interface=127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab
--principal=oryx/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:44:46.956410 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.938652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:33135 (local address 127.13.195.125:41663)
0619 20:44:46.938726 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:46.938728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:46.938858 (+ 130us) server_negotiation.cc:407] Connection header received
0619 20:44:46.939903 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:46.939904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:46.939917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:46.939974 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:46.940910 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.941408 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.942690 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.942883 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.942968 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.946982 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:46.946983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:46.946984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:46.946991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:46.955221 (+ 8230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.955600 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.955601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.955602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.955664 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:46.955877 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:46.955878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:46.955878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:46.955992 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:46.956063 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:46.956224 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 20:44:46.956307 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:44:46.956426 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:46.938335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.13.195.66:33135)
0619 20:44:46.938778 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:46.938789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:46.939799 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:46.940047 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:46.940051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:46.940562 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:46.940812 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.940817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.941589 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.941590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.942594 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:46.942596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:46.942975 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:46.942976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:46.943153 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:46.943878 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:46.943886 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:46.946849 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:46.955308 (+ 8459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.955310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.955310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.955509 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.955731 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:46.955731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:46.955732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:46.955792 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:46.956065 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:46.956066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:46.956108 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:44:46.956186 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:44:46.956316 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":3,"thread_start_us":134,"threads_started":1}
I20260619 20:44:46.956951 21364 heartbeater.cc:347] Connected to a master server at 127.13.195.125:41663
I20260619 20:44:46.956997 21364 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:46.957082 21364 heartbeater.cc:511] Master 127.13.195.125:41663 requested a full tablet report, sending...
I20260619 20:44:46.957379 20860 ts_manager.cc:194] Registered new tserver with Master: 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66:41405)
W20260619 20:44:47.038655 21369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:47.038890 21369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:47.038928 21369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:47.040086 21369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:47.040158 21369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:47.040202 21369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:47.041679 21369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab
--principal=oryx/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:38759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=35353
--tserver_master_addrs=127.13.195.126:45621,127.13.195.125:41663,127.13.195.124:33719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:47.041955 21369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:47.042196 21369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:47.047765 21374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:47.047865 21375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:47.047986 21369 server_base.cc:1034] running on GCE node
W20260619 20:44:47.047786 21377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:47.048277 21369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:47.048550 21369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:47.049726 21369 hybrid_clock.cc:648] HybridClock initialized: now 1781901887049696 us; error 45 us; skew 500 ppm
Jun 19 20:44:47 dist-test-slave-n2td krb5kdc[20217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901887, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:47.052270 21369 init.cc:377] Logged in from keytab as oryx/127.13.195.67@KRBTEST.COM (short username oryx)
I20260619 20:44:47.052728 21369 webserver.cc:460] Webserver started at http://127.13.195.67:35353/ using document root <none> and password file <none>
I20260619 20:44:47.052932 21369 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:47.052984 21369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:47.054286 21369 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:47.055038 21384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:47.055269 21369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:44:47.055341 21369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "138ae6c139c8489c9974c758cbf693fa"
format_stamp: "Formatted at 2026-06-19 20:44:46 on dist-test-slave-n2td"
server_key: "d76c34123a8c20c1c6fefdfc79ae192f"
server_key_iv: "8352963d9bf3c2c2166980296c814a97"
server_key_version: "encryptionkey@0"
I20260619 20:44:47.055668 21369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:47.098929 21369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:47.100942 21369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:47.101048 21369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:47.101228 21369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:47.101567 21369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:47.101600 21369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:47.101653 21369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:47.101678 21369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:47.108359 21369 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:38759
I20260619 20:44:47.108424 21497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:38759 every 8 connection(s)
I20260619 20:44:47.108762 21369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:47.110589 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 21369
Jun 19 20:44:47 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901887, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.124@KRBTEST.COM
Jun 19 20:44:47 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901887, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.125@KRBTEST.COM
Jun 19 20:44:47 dist-test-slave-n2td krb5kdc[20217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901887, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:47.121236 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.109516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.13.195.67:34975)
0619 20:44:47.109996 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.110008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.110993 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.111370 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.111371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.111886 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:47.112168 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.112172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.112916 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.112916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.114010 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.114013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.114626 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.114767 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.115393 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:47.115403 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:47.117442 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:47.119450 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:47.119456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:47.119459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:47.119763 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:47.119990 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:47.119991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:47.119991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:47.120137 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:47.120453 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:47.120460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:47.120620 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.120724 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.120916 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":31,"thread_start_us":165,"threads_started":1}
I20260619 20:44:47.121693 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.109508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:34975 (local address 127.13.195.124:33719)
0619 20:44:47.109560 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.109562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.110088 (+ 526us) server_negotiation.cc:407] Connection header received
0619 20:44:47.111119 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.111121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.111135 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.111169 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.112266 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.112790 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114201 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.114503 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114567 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.117533 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.117534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.117535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.117542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.119368 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.119856 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.119857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.119857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.119917 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.120215 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.120217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.120217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.120363 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.120427 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.121516 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.121601 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:44:47.122315 21498 heartbeater.cc:347] Connected to a master server at 127.13.195.124:33719
I20260619 20:44:47.122421 21498 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:47.122650 21498 heartbeater.cc:511] Master 127.13.195.124:33719 requested a full tablet report, sending...
I20260619 20:44:47.123092 21037 ts_manager.cc:194] Registered new tserver with Master: 138ae6c139c8489c9974c758cbf693fa (127.13.195.67:38759)
I20260619 20:44:47.123842 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.109515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.13.195.67:51665)
0619 20:44:47.109748 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.109790 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.110985 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.111364 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.111368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.111884 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:47.112173 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.112176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.113127 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.113129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.114370 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.114372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.114864 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.114938 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.115956 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:47.115961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:47.118733 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:47.122527 (+ 3794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:47.122528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:47.122529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:47.122766 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:47.123036 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:47.123037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:47.123037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:47.123096 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:47.123451 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:47.123452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:47.123505 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.123571 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.123666 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":218,"thread_start_us":160,"threads_started":1}
I20260619 20:44:47.124166 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.110032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:51665 (local address 127.13.195.125:41663)
0619 20:44:47.110169 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.110171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.110179 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:47.111138 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.111140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.111152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.111187 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.112277 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.113030 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114466 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.114742 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114802 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.118819 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.118820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.118821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.118828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.122143 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.122903 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.122904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.122904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.122963 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.123243 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.123244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.123244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.123383 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.123450 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.123978 (+ 528us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.124068 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260619 20:44:47.124496 21500 heartbeater.cc:347] Connected to a master server at 127.13.195.125:41663
I20260619 20:44:47.124552 21500 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:47.124614 21500 heartbeater.cc:511] Master 127.13.195.125:41663 requested a full tablet report, sending...
I20260619 20:44:47.124872 20860 ts_manager.cc:194] Registered new tserver with Master: 138ae6c139c8489c9974c758cbf693fa (127.13.195.67:38759)
I20260619 20:44:47.125162 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.110179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:48737 (local address 127.13.195.126:45621)
0619 20:44:47.110248 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.110250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.110328 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:44:47.111150 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.111151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.111163 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.111195 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.112276 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.113070 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114309 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.114699 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114759 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.119431 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.119432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.119432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.119439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.121974 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.124313 (+ 2339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.124314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.124314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.124376 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.124596 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.124597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.124597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.124732 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.124783 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.124984 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.125072 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:44:47.125555 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.109861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:48737)
0619 20:44:47.110281 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.110290 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.110988 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.111408 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.111409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.111886 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:47.112183 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.112185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.113152 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.113153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.114217 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.114219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.114837 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.114838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.114953 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.116248 (+ 1295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:47.116252 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:47.119312 (+ 3060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:47.124055 (+ 4743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:47.124056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:47.124057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:47.124252 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:47.124447 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:47.124448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:47.124448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:47.124501 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:47.124816 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:47.124817 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:47.124856 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.125272 (+ 416us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.125369 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":41,"thread_start_us":86,"threads_started":1}
I20260619 20:44:47.125710 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.111626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.13.195.126:45621)
0619 20:44:47.111965 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.111967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.114486 (+ 2519us) server_negotiation.cc:407] Connection header received
0619 20:44:47.114507 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.114507 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.114529 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.114612 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.115560 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.116082 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.117188 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.117498 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.117574 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.120739 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.120740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.120740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.120747 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.123972 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.124662 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.124663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.124663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.124706 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.124901 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.124902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.124902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.125018 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.125102 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.125525 (+ 423us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.125617 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":75,"threads_started":1}
I20260619 20:44:47.126099 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.113642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.13.195.124:33719)
0619 20:44:47.114122 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.114124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.116437 (+ 2313us) server_negotiation.cc:407] Connection header received
0619 20:44:47.116546 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.116547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.116561 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.116603 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.118225 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.118977 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.120050 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.120388 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.120434 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.122232 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.122234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.122235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.122241 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.124147 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.125240 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.125241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.125242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.125309 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.125513 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.125514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.125514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.125635 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.125716 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.125925 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.126003 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":66,"threads_started":1}
I20260619 20:44:47.126340 21499 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45621
I20260619 20:44:47.126392 21499 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:47.126487 21499 heartbeater.cc:511] Master 127.13.195.126:45621 requested a full tablet report, sending...
I20260619 20:44:47.126713 20261 ts_manager.cc:194] Re-registered known tserver with Master: 138ae6c139c8489c9974c758cbf693fa (127.13.195.67:38759)
I20260619 20:44:47.127195 20261 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:48737
I20260619 20:44:47.127957 20261 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:47.130162 20261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:47.130700 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.114106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.13.195.125:41663)
0619 20:44:47.114202 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.114204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.116626 (+ 2422us) server_negotiation.cc:407] Connection header received
0619 20:44:47.116746 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.116746 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.116759 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.116799 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.117500 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.123994 (+ 6494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.126567 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.126864 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.126940 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.128220 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.128222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.128222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.128228 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.129520 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.129894 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.129895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.129895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.129941 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.130159 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.130160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.130161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.130292 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.130386 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.130518 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.130599 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":58,"threads_started":1}
I20260619 20:44:47.137871 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.133872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:41405 (local address 127.0.0.1:33686)
0619 20:44:47.134003 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.134015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.134055 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.134936 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.134937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.134938 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:47.134968 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.134970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.135905 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.135907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.136809 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.136811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.137658 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.137659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.137718 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.137721 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.137768 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.137802 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":30}
I20260619 20:44:47.137988 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.134139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.13.195.66:41405)
0619 20:44:47.134744 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.134748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.134755 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:44:47.134824 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.134826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.134842 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.134943 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:47.135004 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.135737 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.136890 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.137574 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.137648 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.137788 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.137816 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.137855 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":163,"threads_started":1}
I20260619 20:44:47.138751 21523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.134019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:41687 (local address 127.0.0.1:51248)
0619 20:44:47.134553 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.134561 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.134614 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.134819 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.134820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.134822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:47.134872 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.134873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.135808 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.135809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.137081 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.137083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.138493 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.138494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.138565 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.138570 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.138622 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.138659 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":27,"thread_start_us":442,"threads_started":1}
I20260619 20:44:47.139017 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.134154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.13.195.65:41687)
0619 20:44:47.134420 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.134424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.134637 (+ 213us) server_negotiation.cc:407] Connection header received
0619 20:44:47.134708 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.134710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.134724 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.134791 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:47.135026 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.135726 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.137166 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.138215 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.138300 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.138457 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.138851 (+ 394us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.138910 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":133,"threads_started":1}
I20260619 20:44:47.138891 21296 tablet_service.cc:1467] Processing CreateTablet for tablet 839ff0450cae40df9c58145f522c17ea (DEFAULT_TABLE table=test-table [id=73affee4ed96465696585535edc0be80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:47.139317 21296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 839ff0450cae40df9c58145f522c17ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:47.139750 21159 tablet_service.cc:1467] Processing CreateTablet for tablet 839ff0450cae40df9c58145f522c17ea (DEFAULT_TABLE table=test-table [id=73affee4ed96465696585535edc0be80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:47.139986 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.133956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38759 (local address 127.0.0.1:41374)
0619 20:44:47.134461 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.134472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.134535 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.136220 (+ 1685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.136221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.136222 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:47.136256 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.136257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.137390 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.137391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.138596 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.138599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.139716 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.139717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.139798 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.139801 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.139855 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.139908 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":148,"threads_started":1}
I20260619 20:44:47.140147 21159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 839ff0450cae40df9c58145f522c17ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:47.140312 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.134067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.13.195.67:38759)
0619 20:44:47.136008 (+ 1941us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.136013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.136022 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:47.136072 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.136074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.136090 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.136164 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:47.136328 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.137315 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.138685 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.139477 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.139543 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.139678 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.140141 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.140201 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1890,"thread_start_us":147,"threads_started":1}
I20260619 20:44:47.143558 21524 tablet_bootstrap.cc:492] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Bootstrap starting.
I20260619 20:44:47.144040 21525 tablet_bootstrap.cc:492] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Bootstrap starting.
I20260619 20:44:47.144807 21524 tablet_bootstrap.cc:654] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:47.144696 21432 tablet_service.cc:1467] Processing CreateTablet for tablet 839ff0450cae40df9c58145f522c17ea (DEFAULT_TABLE table=test-table [id=73affee4ed96465696585535edc0be80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:47.145241 21525 tablet_bootstrap.cc:654] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:47.145213 21432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 839ff0450cae40df9c58145f522c17ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:47.145807 21524 log.cc:824] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:47.146049 21525 log.cc:824] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:47.146801 21524 tablet_bootstrap.cc:492] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: No bootstrap required, opened a new log
I20260619 20:44:47.146874 21524 ts_tablet_manager.cc:1397] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:44:47.150658 21528 tablet_bootstrap.cc:492] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Bootstrap starting.
I20260619 20:44:47.150575 21524 raft_consensus.cc:348] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.150835 21524 raft_consensus.cc:374] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:47.150882 21524 raft_consensus.cc:729] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0776a7d676f649dcb7e7c469b3148bc3, State: Initialized, Role: FOLLOWER
I20260619 20:44:47.150996 21524 consensus_queue.cc:260] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.151285 21524 ts_tablet_manager.cc:1428] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260619 20:44:47.151463 21363 heartbeater.cc:503] Master 127.13.195.126:45621 was elected leader, sending a full tablet report...
I20260619 20:44:47.152163 21525 tablet_bootstrap.cc:492] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: No bootstrap required, opened a new log
I20260619 20:44:47.152242 21525 ts_tablet_manager.cc:1397] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260619 20:44:47.152575 21528 tablet_bootstrap.cc:654] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:47.153158 21528 log.cc:824] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:47.153990 21528 tablet_bootstrap.cc:492] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: No bootstrap required, opened a new log
I20260619 20:44:47.154043 21528 ts_tablet_manager.cc:1397] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:44:47.155730 21525 raft_consensus.cc:348] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.155880 21525 raft_consensus.cc:374] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:47.155926 21525 raft_consensus.cc:729] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875aa58d46f948a7ab7216a45fb133dc, State: Initialized, Role: FOLLOWER
I20260619 20:44:47.156037 21525 consensus_queue.cc:260] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.156409 21226 heartbeater.cc:503] Master 127.13.195.126:45621 was elected leader, sending a full tablet report...
I20260619 20:44:47.156389 21525 ts_tablet_manager.cc:1428] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 20:44:47.156832 21528 raft_consensus.cc:348] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.156970 21528 raft_consensus.cc:374] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:47.157004 21528 raft_consensus.cc:729] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138ae6c139c8489c9974c758cbf693fa, State: Initialized, Role: FOLLOWER
I20260619 20:44:47.157135 21528 consensus_queue.cc:260] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.157414 21528 ts_tablet_manager.cc:1428] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260619 20:44:47.157528 21499 heartbeater.cc:503] Master 127.13.195.126:45621 was elected leader, sending a full tablet report...
W20260619 20:44:47.189181 21365 tablet.cc:2369] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:47.239058 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.236426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39642 (local address 127.13.195.124:33719)
0619 20:44:47.236481 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.236483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.236494 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:44:47.236601 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.236602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.236623 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.236699 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:47.236853 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.237346 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.238269 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.238696 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.238787 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.238876 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.238942 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.238981 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:44:47.239058 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.236305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.0.0.1:39642)
0619 20:44:47.236422 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.236451 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.236509 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.236706 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.236707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.236708 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:47.236768 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.236774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.237433 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.237434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.238166 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.238168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.238794 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.238795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.238867 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.238870 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.238923 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.238978 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 20:44:47.239547 21056 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 20:44:47.240483 21533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader add908bac93a43b1971ccd05c7ff3a3a. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:47.240620 21533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:44:47.272655 21529 raft_consensus.cc:482] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:47.272847 21529 raft_consensus.cc:504] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.273317 21529 leader_election.cc:288] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687), 138ae6c139c8489c9974c758cbf693fa (127.13.195.67:38759)
I20260619 20:44:47.276304 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.273586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:55441 (local address 127.13.195.67:38759)
0619 20:44:47.273630 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.273632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.273649 (+ 17us) server_negotiation.cc:407] Connection header received
0619 20:44:47.273723 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.273725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.273734 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.273792 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:47.274029 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.274580 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.275492 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.275882 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.275936 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.275942 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.276190 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.276234 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:44:47.276278 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.273510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38759 (local address 127.13.195.66:55441)
0619 20:44:47.273600 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.273612 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.273661 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.273806 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.273807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.273808 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:47.273922 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.273924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.274655 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.274656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.275405 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.275407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.275958 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.275959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.276097 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.276101 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.276158 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.276194 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 20:44:47.276398 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.273586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:53485 (local address 127.13.195.65:41687)
0619 20:44:47.273676 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.273678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.273711 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:44:47.273812 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.273812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.273822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.273879 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:47.274110 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.274634 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.275609 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.275986 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.276080 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.276089 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.276282 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.276328 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 20:44:47.276445 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.273606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:41687 (local address 127.13.195.66:53485)
0619 20:44:47.273643 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:47.273654 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:47.273722 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:47.273902 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:47.273902 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:47.273903 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:47.274032 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.274034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.274716 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.274717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.275521 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:47.275523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.276063 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.276064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:47.276197 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.276201 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:44:47.276258 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:44:47.276295 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 20:44:47.276790 21452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "839ff0450cae40df9c58145f522c17ea" candidate_uuid: "0776a7d676f649dcb7e7c469b3148bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138ae6c139c8489c9974c758cbf693fa" is_pre_election: true
I20260619 20:44:47.276808 21179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "839ff0450cae40df9c58145f522c17ea" candidate_uuid: "0776a7d676f649dcb7e7c469b3148bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875aa58d46f948a7ab7216a45fb133dc" is_pre_election: true
I20260619 20:44:47.276952 21452 raft_consensus.cc:2454] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0776a7d676f649dcb7e7c469b3148bc3 in term 0.
I20260619 20:44:47.276965 21179 raft_consensus.cc:2454] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0776a7d676f649dcb7e7c469b3148bc3 in term 0.
I20260619 20:44:47.277197 21250 leader_election.cc:302] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0776a7d676f649dcb7e7c469b3148bc3, 875aa58d46f948a7ab7216a45fb133dc; no voters:
I20260619 20:44:47.277385 21529 raft_consensus.cc:2781] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:47.277441 21529 raft_consensus.cc:482] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:47.277472 21529 raft_consensus.cc:3037] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:47.278391 21529 raft_consensus.cc:504] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.278573 21529 leader_election.cc:288] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [CANDIDATE]: Term 1 election: Requested vote from peers 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687), 138ae6c139c8489c9974c758cbf693fa (127.13.195.67:38759)
I20260619 20:44:47.278710 21179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "839ff0450cae40df9c58145f522c17ea" candidate_uuid: "0776a7d676f649dcb7e7c469b3148bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875aa58d46f948a7ab7216a45fb133dc"
I20260619 20:44:47.278750 21452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "839ff0450cae40df9c58145f522c17ea" candidate_uuid: "0776a7d676f649dcb7e7c469b3148bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138ae6c139c8489c9974c758cbf693fa"
I20260619 20:44:47.278795 21179 raft_consensus.cc:3037] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:47.278839 21452 raft_consensus.cc:3037] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:47.279712 21179 raft_consensus.cc:2454] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0776a7d676f649dcb7e7c469b3148bc3 in term 1.
I20260619 20:44:47.279747 21452 raft_consensus.cc:2454] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0776a7d676f649dcb7e7c469b3148bc3 in term 1.
I20260619 20:44:47.279883 21250 leader_election.cc:302] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0776a7d676f649dcb7e7c469b3148bc3, 875aa58d46f948a7ab7216a45fb133dc; no voters:
I20260619 20:44:47.279964 21529 raft_consensus.cc:2781] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:47.280133 21529 raft_consensus.cc:686] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 LEADER]: Becoming Leader. State: Replica: 0776a7d676f649dcb7e7c469b3148bc3, State: Running, Role: LEADER
I20260619 20:44:47.280236 21529 consensus_queue.cc:237] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } }
I20260619 20:44:47.281014 20261 catalog_manager.cc:5495] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66). New cstate: current_term: 1 leader_uuid: "0776a7d676f649dcb7e7c469b3148bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:47.281538 20821 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: NON_VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: true } } }
I20260619 20:44:47.281643 20821 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:47.282474 21010 raft_consensus.cc:1050] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: attempting to promote NON_VOTER 61e2db694ada433087397526ac6ce524 to VOTER
I20260619 20:44:47.282610 21010 consensus_queue.cc:237] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:47.283037 21056 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 LEARNER]: Refusing update from remote peer add908bac93a43b1971ccd05c7ff3a3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 20:44:47.283226 21540 consensus_queue.cc:1035] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 20:44:47.283295 20870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Refusing update from remote peer add908bac93a43b1971ccd05c7ff3a3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 20:44:47.283493 21540 consensus_queue.cc:1035] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 20:44:47.283895 20821 raft_consensus.cc:2932] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 61e2db694ada433087397526ac6ce524 (127.13.195.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.284077 21056 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 61e2db694ada433087397526ac6ce524 (127.13.195.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.285095 20821 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.285179 20821 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:47.285140 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.285226 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:47.285247 20821 sys_catalog.cc:455] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.285303 20821 sys_catalog.cc:458] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:47.285393 21533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.285472 21533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:44:47.285410 20870 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 61e2db694ada433087397526ac6ce524 (127.13.195.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.286228 21542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "add908bac93a43b1971ccd05c7ff3a3a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:47.286316 21542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:44:47.290592 21228 tablet.cc:2369] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:47.317531 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.315332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.13.195.66:41405)
0619 20:44:47.315435 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.315438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.315447 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:47.315483 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.315483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.315500 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.315579 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:47.315706 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.316192 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.316886 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.317073 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.317126 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.317248 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:47.317346 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:47.317438 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.317450 (+ 12us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.317468 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:44:47.319901 21179 raft_consensus.cc:1261] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 1 FOLLOWER]: Refusing update from remote peer 0776a7d676f649dcb7e7c469b3148bc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:47.319936 21452 raft_consensus.cc:1261] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 1 FOLLOWER]: Refusing update from remote peer 0776a7d676f649dcb7e7c469b3148bc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:47.320302 21529 consensus_queue.cc:1035] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:47.320518 21538 consensus_queue.cc:1035] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:47.325281 21546 mvcc.cc:205] Tried to move back new op lower bound from 7298670130458804224 to 7298670130300387328. Current Snapshot: MvccSnapshot[applied={T|T < 7298670130458804224}]
I20260619 20:44:47.327204 20261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:47.327315 20261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:47.328842 20261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P add908bac93a43b1971ccd05c7ff3a3a: Sending DeleteTablet for 3 replicas of tablet 839ff0450cae40df9c58145f522c17ea
I20260619 20:44:47.329152 21296 tablet_service.cc:1514] Processing DeleteTablet for tablet 839ff0450cae40df9c58145f522c17ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:47 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:33686
I20260619 20:44:47.329351 21432 tablet_service.cc:1514] Processing DeleteTablet for tablet 839ff0450cae40df9c58145f522c17ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:47 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:41374
I20260619 20:44:47.329495 21553 tablet_replica.cc:330] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: stopping tablet replica
I20260619 20:44:47.329623 21553 raft_consensus.cc:2229] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:47.329739 21553 raft_consensus.cc:2258] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:47.329836 21159 tablet_service.cc:1514] Processing DeleteTablet for tablet 839ff0450cae40df9c58145f522c17ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:47 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:51248
I20260619 20:44:47.330276 21554 tablet_replica.cc:330] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: stopping tablet replica
I20260619 20:44:47.330261 21553 ts_tablet_manager.cc:1905] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:47.330379 21554 raft_consensus.cc:2229] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:47.330516 21554 raft_consensus.cc:2258] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:47.331032 21554 ts_tablet_manager.cc:1905] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:47.331565 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 20230
I20260619 20:44:47.333153 21553 ts_tablet_manager.cc:1918] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:47.333256 21553 log.cc:1196] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/839ff0450cae40df9c58145f522c17ea
I20260619 20:44:47.333600 21553 ts_tablet_manager.cc:1939] T 839ff0450cae40df9c58145f522c17ea P 0776a7d676f649dcb7e7c469b3148bc3: Deleting consensus metadata
I20260619 20:44:47.334200 21554 ts_tablet_manager.cc:1918] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:47.334297 21554 log.cc:1196] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/839ff0450cae40df9c58145f522c17ea
I20260619 20:44:47.334647 21554 ts_tablet_manager.cc:1939] T 839ff0450cae40df9c58145f522c17ea P 138ae6c139c8489c9974c758cbf693fa: Deleting consensus metadata
W20260619 20:44:47.339190 21386 connection.cc:535] server connection from 127.0.0.1:41374 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 20:44:47.339401 21249 connection.cc:535] server connection from 127.0.0.1:33686 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:44:47.340932 21555 tablet_replica.cc:330] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: stopping tablet replica
I20260619 20:44:47.341068 21555 raft_consensus.cc:2229] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:47.341198 21555 raft_consensus.cc:2258] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:47.342087 21555 ts_tablet_manager.cc:1905] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:47.342931 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.331338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.13.195.124:33719)
0619 20:44:47.331389 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.331391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.335075 (+ 3684us) server_negotiation.cc:407] Connection header received
0619 20:44:47.335157 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.335158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.335170 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.335219 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.335892 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.336464 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.337311 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.337548 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.337633 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.339756 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.339757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.339757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.339765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.341544 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.342023 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.342025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.342025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.342087 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.342335 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.342336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.342336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.342501 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.342598 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.342749 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.342828 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:44:47.343798 21555 ts_tablet_manager.cc:1918] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:47.343875 21555 log.cc:1196] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/839ff0450cae40df9c58145f522c17ea
I20260619 20:44:47.344161 21555 ts_tablet_manager.cc:1939] T 839ff0450cae40df9c58145f522c17ea P 875aa58d46f948a7ab7216a45fb133dc: Deleting consensus metadata
W20260619 20:44:47.344542 21115 connection.cc:441] server connection from 127.0.0.1:51248 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51248 (request call id 1) could send its response
I20260619 20:44:47.349133 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.340789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.13.195.125:41663)
0619 20:44:47.340841 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.340843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.341251 (+ 408us) server_negotiation.cc:407] Connection header received
0619 20:44:47.341407 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.341409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.341424 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.341498 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.342937 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.343603 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.344748 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.344986 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.345080 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.346376 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.346378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.346379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.346386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.347873 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.348260 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.348261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.348262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.348331 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.348547 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.348547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.348548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.348676 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.348784 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.348921 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.349010 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:44:47.355636 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:47.340996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39662 (local address 127.13.195.124:33719)
0619 20:44:47.341035 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:47.341037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:47.349152 (+ 8115us) server_negotiation.cc:407] Connection header received
0619 20:44:47.349394 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:47.349395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:47.349408 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:47.349451 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:47.350034 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.350652 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.351421 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:47.351645 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:47.351738 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:47.353004 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:47.353005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:47.353006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:47.353014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:47.354357 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.354795 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.354796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.354796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.354867 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:47.355095 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:47.355095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:47.355096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:47.355212 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:47.355326 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:47.355446 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 20:44:47.355527 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:44:48.324949 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.324630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:36785)
0619 20:44:48.324782 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.324884 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
W20260619 20:44:48.325217 21226 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:45621 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:48.327370 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.327130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:47535)
0619 20:44:48.327258 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.327332 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":52,"threads_started":1}
I20260619 20:44:48.327379 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.327158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:44109)
0619 20:44:48.327284 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.327345 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260619 20:44:48.327580 21499 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:45621 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
W20260619 20:44:48.327605 21363 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:45621 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:48.829450 21567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader add908bac93a43b1971ccd05c7ff3a3a)
I20260619 20:44:48.829638 21567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:48.829964 21567 leader_election.cc:288] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621), 18c9747abdc943c9961bf7f236b73562 (127.13.195.125:41663)
I20260619 20:44:48.829969 21568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader add908bac93a43b1971ccd05c7ff3a3a)
I20260619 20:44:48.830106 21568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:48.830318 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.830096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52538)
0619 20:44:48.830222 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.830276 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":44,"threads_started":1}
W20260619 20:44:48.830458 21032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:48.830688 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.830502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52550)
0619 20:44:48.830581 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.830644 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63,"thread_start_us":44,"threads_started":1}
I20260619 20:44:48.830899 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.830808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52564)
0619 20:44:48.830843 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.830868 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 20:44:48.830899 20846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
W20260619 20:44:48.830981 21032 leader_election.cc:334] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621): Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:48.830765 21568 leader_election.cc:288] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621), 61e2db694ada433087397526ac6ce524 (127.13.195.124:33719)
I20260619 20:44:48.831379 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.831290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52578)
0619 20:44:48.831323 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.831348 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 20:44:48.831452 20846 leader_election.cc:334] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621): Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:48.833666 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.830174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.13.195.125:41663)
0619 20:44:48.830276 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:48.830278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:48.830424 (+ 146us) server_negotiation.cc:407] Connection header received
0619 20:44:48.830565 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:48.830566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:48.830578 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:48.830638 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:48.831055 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.831839 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.832953 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.833326 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.833413 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:48.833421 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:48.833566 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:44:48.833606 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":46,"threads_started":1}
I20260619 20:44:48.833669 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.830121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:41663 (local address 127.0.0.1:40094)
0619 20:44:48.830325 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.830338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:48.830476 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:48.830690 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:48.830692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:48.830694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:48.830756 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:48.830758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.831933 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.831934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:48.832865 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:48.832868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.833410 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.833411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:48.833472 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:48.833476 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:44:48.833558 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:44:48.833603 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":49,"threads_started":1}
I20260619 20:44:48.833770 21575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.830920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.13.195.124:33719)
0619 20:44:48.831114 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:48.831116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:48.831124 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:48.831146 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:48.831146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:48.831155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:48.831257 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:48.831384 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.832048 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.833051 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.833416 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.833511 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:48.833527 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:48.833662 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 20:44:48.833702 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260619 20:44:48.833770 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.830695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:33719 (local address 127.0.0.1:39670)
0619 20:44:48.830850 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.830859 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:48.830908 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:48.831272 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:48.831273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:48.831275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:48.831311 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:48.831312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.832133 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.832134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:48.832937 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:48.832938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:48.833511 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:48.833512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:48.833579 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:48.833582 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:48.833655 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:44:48.833702 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":43,"threads_started":1}
I20260619 20:44:48.834052 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e2db694ada433087397526ac6ce524" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "18c9747abdc943c9961bf7f236b73562" is_pre_election: true
I20260619 20:44:48.834106 21056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18c9747abdc943c9961bf7f236b73562" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "61e2db694ada433087397526ac6ce524" is_pre_election: true
I20260619 20:44:48.834156 20870 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61e2db694ada433087397526ac6ce524 in term 1.
I20260619 20:44:48.834203 21056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c9747abdc943c9961bf7f236b73562 in term 1.
I20260619 20:44:48.834323 21030 leader_election.cc:302] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18c9747abdc943c9961bf7f236b73562, 61e2db694ada433087397526ac6ce524; no voters: add908bac93a43b1971ccd05c7ff3a3a
I20260619 20:44:48.834324 20845 leader_election.cc:302] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18c9747abdc943c9961bf7f236b73562, 61e2db694ada433087397526ac6ce524; no voters: add908bac93a43b1971ccd05c7ff3a3a
I20260619 20:44:48.834453 21568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 20:44:48.834453 21567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 20:44:48.834496 21568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Starting leader election (detected failure of leader add908bac93a43b1971ccd05c7ff3a3a)
I20260619 20:44:48.834496 21567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Starting leader election (detected failure of leader add908bac93a43b1971ccd05c7ff3a3a)
I20260619 20:44:48.834529 21567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:44:48.834529 21568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:44:48.835405 21568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:48.835405 21567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:48.835538 21567 leader_election.cc:288] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [CANDIDATE]: Term 2 election: Requested vote from peers add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621), 18c9747abdc943c9961bf7f236b73562 (127.13.195.125:41663)
I20260619 20:44:48.835695 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.835604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52582)
0619 20:44:48.835639 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.835668 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 20:44:48.835712 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e2db694ada433087397526ac6ce524" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "18c9747abdc943c9961bf7f236b73562"
I20260619 20:44:48.835774 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.835623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52586)
0619 20:44:48.835715 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.835744 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 20:44:48.835783 20870 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 61e2db694ada433087397526ac6ce524 in current term 2: Already voted for candidate 18c9747abdc943c9961bf7f236b73562 in this term.
I20260619 20:44:48.835969 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.835878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52598)
0619 20:44:48.835927 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.835948 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:44:48.835973 21056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18c9747abdc943c9961bf7f236b73562" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "61e2db694ada433087397526ac6ce524"
I20260619 20:44:48.836045 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:48.835954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52604)
0619 20:44:48.835995 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:48.836021 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 20:44:48.836046 21056 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 18c9747abdc943c9961bf7f236b73562 in current term 2: Already voted for candidate 61e2db694ada433087397526ac6ce524 in this term.
W20260619 20:44:48.836117 21032 leader_election.cc:334] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621): Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
W20260619 20:44:48.836128 20846 leader_election.cc:334] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621): Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:48.836148 21032 leader_election.cc:302] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e2db694ada433087397526ac6ce524; no voters: 18c9747abdc943c9961bf7f236b73562, add908bac93a43b1971ccd05c7ff3a3a
I20260619 20:44:48.836009 21568 leader_election.cc:288] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 2 election: Requested vote from peers add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621), 61e2db694ada433087397526ac6ce524 (127.13.195.124:33719)
I20260619 20:44:48.836239 21567 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 20:44:48.836268 20845 leader_election.cc:302] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18c9747abdc943c9961bf7f236b73562; no voters: 61e2db694ada433087397526ac6ce524, add908bac93a43b1971ccd05c7ff3a3a
I20260619 20:44:48.836344 21568 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 20:44:49.325994 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:49.325768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:53693)
0619 20:44:49.325894 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:49.325934 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260619 20:44:49.328169 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:49.327919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:57157)
0619 20:44:49.328087 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:49.328130 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260619 20:44:49.328361 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:49.327920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:43975)
0619 20:44:49.328270 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:49.328322 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":98,"threads_started":1}
I20260619 20:44:50.326810 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.326555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:56557)
0619 20:44:50.326684 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.326751 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260619 20:44:50.328886 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.328689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:44297)
0619 20:44:50.328806 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.328842 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260619 20:44:50.328986 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.328737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:48761)
0619 20:44:50.328871 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.328908 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260619 20:44:50.806605 21583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:50.806706 21583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:50.806957 21583 leader_election.cc:288] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621), 61e2db694ada433087397526ac6ce524 (127.13.195.124:33719)
I20260619 20:44:50.807168 21056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18c9747abdc943c9961bf7f236b73562" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "61e2db694ada433087397526ac6ce524" is_pre_election: true
I20260619 20:44:50.807283 21056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c9747abdc943c9961bf7f236b73562 in term 2.
I20260619 20:44:50.807344 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.807096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52878)
0619 20:44:50.807229 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.807288 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260619 20:44:50.807507 20845 leader_election.cc:302] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c9747abdc943c9961bf7f236b73562, 61e2db694ada433087397526ac6ce524; no voters:
I20260619 20:44:50.807617 21583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 20:44:50.807762 21583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:50.807806 21583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:44:50.808104 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.807907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52882)
0619 20:44:50.808009 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.808051 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 20:44:50.808219 20846 leader_election.cc:334] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621): Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:50.808802 21583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:50.808959 21583 leader_election.cc:288] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 3 election: Requested vote from peers add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621), 61e2db694ada433087397526ac6ce524 (127.13.195.124:33719)
I20260619 20:44:50.809113 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.809028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52898)
0619 20:44:50.809060 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.809084 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:44:50.809204 21056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18c9747abdc943c9961bf7f236b73562" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "61e2db694ada433087397526ac6ce524"
I20260619 20:44:50.809405 21056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:44:50.809474 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.809375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52910)
0619 20:44:50.809422 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.809449 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:44:50.809567 20846 leader_election.cc:334] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621): Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
I20260619 20:44:50.810137 21056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c9747abdc943c9961bf7f236b73562 in term 3.
I20260619 20:44:50.810305 20845 leader_election.cc:302] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18c9747abdc943c9961bf7f236b73562, 61e2db694ada433087397526ac6ce524; no voters: add908bac93a43b1971ccd05c7ff3a3a
I20260619 20:44:50.810464 21583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 20:44:50.810568 21583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [term 3 LEADER]: Becoming Leader. State: Replica: 18c9747abdc943c9961bf7f236b73562, State: Running, Role: LEADER
I20260619 20:44:50.810663 21583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } }
I20260619 20:44:50.811060 21586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18c9747abdc943c9961bf7f236b73562. Latest consensus state: current_term: 3 leader_uuid: "18c9747abdc943c9961bf7f236b73562" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:50.811143 21586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:50.811324 21588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:50.811746 21588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:50.811863 21588 catalog_manager.cc:1259] Loaded cluster ID: 844152427beb4df6be97f5b808e7bf76
I20260619 20:44:50.811898 21588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:50.812084 21588 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:50.812178 21588 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Loaded TSK: 0
I20260619 20:44:50.812321 21588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:44:50.844362 20859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:50.845964 20859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:50.846992 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.846868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52916)
0619 20:44:50.846925 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.846957 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:44:50.847046 21056 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [term 3 FOLLOWER]: Refusing update from remote peer 18c9747abdc943c9961bf7f236b73562: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 20:44:50.847375 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.847241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.0.0.1:52932)
0619 20:44:50.847274 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.847306 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:44:50.847311 21586 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260619 20:44:50.847463 20846 consensus_peers.cc:487] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 -> Peer add908bac93a43b1971ccd05c7ff3a3a (127.13.195.126:45621): Couldn't send request to peer add908bac93a43b1971ccd05c7ff3a3a. Status: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 20:44:50.848099 21590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18c9747abdc943c9961bf7f236b73562. Latest consensus state: current_term: 3 leader_uuid: "18c9747abdc943c9961bf7f236b73562" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:50.848210 21590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:44:50.848626 21586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "18c9747abdc943c9961bf7f236b73562" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:50.848722 21586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:50.848752 21590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "18c9747abdc943c9961bf7f236b73562" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "add908bac93a43b1971ccd05c7ff3a3a" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45621 } } peers { permanent_uuid: "18c9747abdc943c9961bf7f236b73562" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 41663 } attrs { promote: false } } peers { permanent_uuid: "61e2db694ada433087397526ac6ce524" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 33719 } attrs { promote: false } } }
I20260619 20:44:50.848903 21590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e2db694ada433087397526ac6ce524 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:44:50.854007 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.850697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:41687 (local address 127.0.0.1:51250)
0619 20:44:50.850826 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.850836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:50.850913 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:50.851199 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:50.851200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:50.851201 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:50.851236 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.851237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.852183 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.852184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.852941 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.852942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.853776 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.853777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.853846 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.853852 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:44:50.853882 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 20:44:50.853922 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260619 20:44:50.854189 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.850850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.13.195.66:41405)
0619 20:44:50.851241 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:50.851243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:50.851251 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:50.851284 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:50.851285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:50.851297 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:50.851358 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:50.851463 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.851944 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.853466 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.853865 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.853901 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.853910 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:50.854079 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 20:44:50.854125 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":58,"threads_started":1}
I20260619 20:44:50.854305 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.850675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38759 (local address 127.0.0.1:41380)
0619 20:44:50.850720 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.850729 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:50.850777 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:50.851140 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:50.851141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:50.851143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:50.851199 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.851201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.852154 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.852155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.853227 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.853229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.854116 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.854117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.854177 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.854180 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:50.854208 (+ 28us) client_negotiation.cc:240] Negotiation successful
0619 20:44:50.854247 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":29}
I20260619 20:44:50.854656 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.850960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.13.195.65:41687)
0619 20:44:50.851074 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:50.851076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:50.851084 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:50.851108 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:50.851109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:50.851117 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:50.851297 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:50.851335 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.852081 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.853049 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.853651 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.854502 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.854525 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:50.854545 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 20:44:50.854591 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260619 20:44:50.854698 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.850879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.13.195.67:38759)
0619 20:44:50.850988 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:50.850990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:50.850999 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:50.851026 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:50.851027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:50.851038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:50.851133 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:50.851282 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.852076 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.853309 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.854015 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.854560 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.854572 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:50.854590 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 20:44:50.854628 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260619 20:44:50.854188 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.850717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:41405 (local address 127.0.0.1:33718)
0619 20:44:50.850987 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.850996 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:50.851035 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:50.851359 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:50.851360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:50.851362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:50.851410 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.851411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.852040 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.852040 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.853372 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.853373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.853957 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.853958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.853999 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.854001 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:44:50.854069 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:44:50.854110 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":63,"threads_started":1}
I20260619 20:44:50.854924 21432 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad07207c150494aadc05df1cf95c6ae (DEFAULT_TABLE table=test-table [id=97cfd1f3a0cd47088c2454e18bea37e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:50.855067 21432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad07207c150494aadc05df1cf95c6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:50.856406 21600 tablet_bootstrap.cc:492] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Bootstrap starting.
I20260619 20:44:50.856448 21159 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad07207c150494aadc05df1cf95c6ae (DEFAULT_TABLE table=test-table [id=97cfd1f3a0cd47088c2454e18bea37e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:50.856575 21159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad07207c150494aadc05df1cf95c6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:50.856844 21600 tablet_bootstrap.cc:654] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:50.857455 21600 tablet_bootstrap.cc:492] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: No bootstrap required, opened a new log
I20260619 20:44:50.857498 21600 ts_tablet_manager.cc:1397] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:50.857649 21600 raft_consensus.cc:348] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.857709 21600 raft_consensus.cc:374] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:50.857722 21600 raft_consensus.cc:729] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138ae6c139c8489c9974c758cbf693fa, State: Initialized, Role: FOLLOWER
I20260619 20:44:50.857792 21600 consensus_queue.cc:260] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.857968 21602 tablet_bootstrap.cc:492] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc: Bootstrap starting.
I20260619 20:44:50.857976 21600 ts_tablet_manager.cc:1428] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:50.858053 21296 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad07207c150494aadc05df1cf95c6ae (DEFAULT_TABLE table=test-table [id=97cfd1f3a0cd47088c2454e18bea37e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:50.858189 21296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad07207c150494aadc05df1cf95c6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:50.858789 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.858251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:37457)
0619 20:44:50.858704 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.858744 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":89,"threads_started":1}
I20260619 20:44:50.859099 21500 heartbeater.cc:503] Master 127.13.195.125:41663 was elected leader, sending a full tablet report...
I20260619 20:44:50.859230 21602 tablet_bootstrap.cc:654] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:50.859299 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.859218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:56077)
0619 20:44:50.859251 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.859275 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 20:44:50.859884 21606 tablet_bootstrap.cc:492] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: Bootstrap starting.
I20260619 20:44:50.859923 21602 tablet_bootstrap.cc:492] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc: No bootstrap required, opened a new log
I20260619 20:44:50.859968 21602 ts_tablet_manager.cc:1397] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:44:50.860208 21602 raft_consensus.cc:348] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.860268 21602 raft_consensus.cc:374] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:50.860282 21602 raft_consensus.cc:729] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875aa58d46f948a7ab7216a45fb133dc, State: Initialized, Role: FOLLOWER
I20260619 20:44:50.860363 21606 tablet_bootstrap.cc:654] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:50.860373 21602 consensus_queue.cc:260] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.860585 21602 ts_tablet_manager.cc:1428] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:50.860898 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.860690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:51275)
0619 20:44:50.860820 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.860853 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260619 20:44:50.861068 21227 heartbeater.cc:503] Master 127.13.195.125:41663 was elected leader, sending a full tablet report...
I20260619 20:44:50.861239 21606 tablet_bootstrap.cc:492] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: No bootstrap required, opened a new log
I20260619 20:44:50.861274 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.861120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:44401)
0619 20:44:50.861220 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.861244 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260619 20:44:50.861294 21606 ts_tablet_manager.cc:1397] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:50.861408 21606 raft_consensus.cc:348] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.861477 21606 raft_consensus.cc:374] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:50.861496 21606 raft_consensus.cc:729] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0776a7d676f649dcb7e7c469b3148bc3, State: Initialized, Role: FOLLOWER
I20260619 20:44:50.861570 21606 consensus_queue.cc:260] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.861696 21606 ts_tablet_manager.cc:1428] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 20:44:50.861876 21501 tablet.cc:2369] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:50.862193 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.861977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:35873)
0619 20:44:50.862119 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.862156 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260619 20:44:50.862381 21364 heartbeater.cc:503] Master 127.13.195.125:41663 was elected leader, sending a full tablet report...
I20260619 20:44:50.862679 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.862593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:43649)
0619 20:44:50.862625 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.862652 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:44:50.895732 21603 raft_consensus.cc:482] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:50.895829 21603 raft_consensus.cc:504] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.896236 21603 leader_election.cc:288] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687), 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66:41405)
I20260619 20:44:50.899415 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.896389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:41687 (local address 127.13.195.67:53143)
0619 20:44:50.896631 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.896640 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:50.896680 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:50.896880 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:50.896880 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:50.896882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:50.896931 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.896932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.897669 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.897670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.898564 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.898568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.899147 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.899147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.899218 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.899221 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:50.899292 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:44:50.899331 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":2,"thread_start_us":194,"threads_started":1}
I20260619 20:44:50.899468 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.896705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:53143 (local address 127.13.195.65:41687)
0619 20:44:50.896747 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:50.896749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:50.896756 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:44:50.896783 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:50.896784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:50.896796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:50.896866 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:50.897004 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.897578 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.898662 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.899046 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.899146 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.899158 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:50.899300 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 20:44:50.899338 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:44:50.899518 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.896474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:51529 (local address 127.13.195.66:41405)
0619 20:44:50.896588 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:50.896591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:50.896601 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:50.896626 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:50.896627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:50.896638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:50.896694 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:50.896941 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.897528 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.898654 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.899041 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.899149 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.899160 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:50.899394 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 20:44:50.899442 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 20:44:50.899519 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.896383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:41405 (local address 127.13.195.67:51529)
0619 20:44:50.896431 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.896443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:50.896492 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:50.896708 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:50.896710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:50.896713 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:50.896848 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.896850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.897623 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.897624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.898528 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:50.898531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.899150 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.899151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:50.899285 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.899288 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:50.899373 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:44:50.899418 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 20:44:50.899926 21179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad07207c150494aadc05df1cf95c6ae" candidate_uuid: "138ae6c139c8489c9974c758cbf693fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875aa58d46f948a7ab7216a45fb133dc" is_pre_election: true
I20260619 20:44:50.900063 21179 raft_consensus.cc:2454] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 138ae6c139c8489c9974c758cbf693fa in term 0.
I20260619 20:44:50.899961 21316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad07207c150494aadc05df1cf95c6ae" candidate_uuid: "138ae6c139c8489c9974c758cbf693fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0776a7d676f649dcb7e7c469b3148bc3" is_pre_election: true
I20260619 20:44:50.900142 21316 raft_consensus.cc:2454] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 138ae6c139c8489c9974c758cbf693fa in term 0.
I20260619 20:44:50.900349 21387 leader_election.cc:302] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0776a7d676f649dcb7e7c469b3148bc3, 138ae6c139c8489c9974c758cbf693fa; no voters:
I20260619 20:44:50.900496 21603 raft_consensus.cc:2781] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:50.900538 21603 raft_consensus.cc:482] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:50.900564 21603 raft_consensus.cc:3037] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:50.901328 21603 raft_consensus.cc:504] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.901490 21603 leader_election.cc:288] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [CANDIDATE]: Term 1 election: Requested vote from peers 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687), 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66:41405)
I20260619 20:44:50.901650 21179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad07207c150494aadc05df1cf95c6ae" candidate_uuid: "138ae6c139c8489c9974c758cbf693fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875aa58d46f948a7ab7216a45fb133dc"
I20260619 20:44:50.901635 21316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad07207c150494aadc05df1cf95c6ae" candidate_uuid: "138ae6c139c8489c9974c758cbf693fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0776a7d676f649dcb7e7c469b3148bc3"
I20260619 20:44:50.901715 21179 raft_consensus.cc:3037] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:50.901737 21316 raft_consensus.cc:3037] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:50.902289 21179 raft_consensus.cc:2454] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 138ae6c139c8489c9974c758cbf693fa in term 1.
I20260619 20:44:50.902491 21386 leader_election.cc:302] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138ae6c139c8489c9974c758cbf693fa, 875aa58d46f948a7ab7216a45fb133dc; no voters:
I20260619 20:44:50.902644 21603 raft_consensus.cc:2781] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:50.902678 21316 raft_consensus.cc:2454] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 138ae6c139c8489c9974c758cbf693fa in term 1.
I20260619 20:44:50.902815 21603 raft_consensus.cc:686] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 1 LEADER]: Becoming Leader. State: Replica: 138ae6c139c8489c9974c758cbf693fa, State: Running, Role: LEADER
I20260619 20:44:50.902884 21603 consensus_queue.cc:237] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } }
I20260619 20:44:50.903275 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.903147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:36353)
0619 20:44:50.903209 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.903238 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:44:50.903620 20859 catalog_manager.cc:5495] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 138ae6c139c8489c9974c758cbf693fa (127.13.195.67). New cstate: current_term: 1 leader_uuid: "138ae6c139c8489c9974c758cbf693fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ae6c139c8489c9974c758cbf693fa" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:50.912358 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.910077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.13.195.67:38759)
0619 20:44:50.910150 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:50.910152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:50.910170 (+ 18us) server_negotiation.cc:407] Connection header received
0619 20:44:50.910237 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:50.910238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:50.910253 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:50.910318 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:50.910413 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.910933 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.911631 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:50.911809 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:50.911861 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:50.911985 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:50.912086 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:50.912138 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:50.912227 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 20:44:50.912244 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:44:50.913545 21179 raft_consensus.cc:1261] T 4ad07207c150494aadc05df1cf95c6ae P 875aa58d46f948a7ab7216a45fb133dc [term 1 FOLLOWER]: Refusing update from remote peer 138ae6c139c8489c9974c758cbf693fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:50.913718 21316 raft_consensus.cc:1261] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 FOLLOWER]: Refusing update from remote peer 138ae6c139c8489c9974c758cbf693fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:50.913820 21603 consensus_queue.cc:1035] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "875aa58d46f948a7ab7216a45fb133dc" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:50.914023 21603 consensus_queue.cc:1035] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0776a7d676f649dcb7e7c469b3148bc3" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:50.914717 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.914604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:60065)
0619 20:44:50.914648 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.914680 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 20:44:50.915601 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.915240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:33235)
0619 20:44:50.915518 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.915564 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259}
I20260619 20:44:50.915860 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.915658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:52839)
0619 20:44:50.915802 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.915831 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260619 20:44:50.915895 21618 mvcc.cc:205] Tried to move back new op lower bound from 7298670145179619328 to 7298670145138651136. Current Snapshot: MvccSnapshot[applied={T|T < 7298670145179619328}]
I20260619 20:44:50.916760 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.916657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.67:57447)
0619 20:44:50.916706 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.916735 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:44:50.916930 21620 mvcc.cc:205] Tried to move back new op lower bound from 7298670145179619328 to 7298670145138651136. Current Snapshot: MvccSnapshot[applied={T|T < 7298670145179619328}]
I20260619 20:44:50.917249 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.917161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.66:34407)
0619 20:44:50.917197 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.917221 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 20:44:50.917758 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:50.917659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45621 (local address 127.13.195.65:51835)
0619 20:44:50.917696 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.917728 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 20:44:50.917904 20860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:50.917989 20860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:50.919322 20860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18c9747abdc943c9961bf7f236b73562: Sending DeleteTablet for 3 replicas of tablet 4ad07207c150494aadc05df1cf95c6ae
I20260619 20:44:50.919596 21159 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad07207c150494aadc05df1cf95c6ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:50 UTC) from {username='oryx', principal='oryx/127.13.195.125@KRBTEST.COM'} at 127.0.0.1:51250
I20260619 20:44:50.919740 21432 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad07207c150494aadc05df1cf95c6ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:50 UTC) from {username='oryx', principal='oryx/127.13.195.125@KRBTEST.COM'} at 127.0.0.1:41380
I20260619 20:44:50.919775 21296 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad07207c150494aadc05df1cf95c6ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:50 UTC) from {username='oryx', principal='oryx/127.13.195.125@KRBTEST.COM'} at 127.0.0.1:33718
I20260619 20:44:50.919790 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 21007
I20260619 20:44:50.922554 21627 tablet_replica.cc:330] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: stopping tablet replica
I20260619 20:44:50.922627 21627 raft_consensus.cc:2229] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:50.922676 21627 raft_consensus.cc:2258] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:50.922772 21628 tablet_replica.cc:330] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: stopping tablet replica
I20260619 20:44:50.922847 21628 raft_consensus.cc:2229] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:50.922982 21628 raft_consensus.cc:2258] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:50.923296 21628 ts_tablet_manager.cc:1905] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:50.923856 21627 ts_tablet_manager.cc:1905] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:50.924957 21628 ts_tablet_manager.cc:1918] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:50.925088 21628 log.cc:1196] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/4ad07207c150494aadc05df1cf95c6ae
I20260619 20:44:50.925403 21628 ts_tablet_manager.cc:1939] T 4ad07207c150494aadc05df1cf95c6ae P 138ae6c139c8489c9974c758cbf693fa: Deleting consensus metadata
I20260619 20:44:50.925784 20847 catalog_manager.cc:4841] TS 138ae6c139c8489c9974c758cbf693fa (127.13.195.67:38759): tablet 4ad07207c150494aadc05df1cf95c6ae (table test-table [id=97cfd1f3a0cd47088c2454e18bea37e9]) successfully deleted
I20260619 20:44:50.925866 21627 ts_tablet_manager.cc:1918] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:50.925918 21627 log.cc:1196] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/4ad07207c150494aadc05df1cf95c6ae
I20260619 20:44:50.926182 21627 ts_tablet_manager.cc:1939] T 4ad07207c150494aadc05df1cf95c6ae P 0776a7d676f649dcb7e7c469b3148bc3: Deleting consensus metadata
W20260619 20:44:50.926388 21596 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:50.925818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:41687 (local address 127.0.0.1:51256)
0619 20:44:50.925919 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:50.925931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:50.925972 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:50.926338 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:41687: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 20:44:50.926489 20846 catalog_manager.cc:4841] TS 0776a7d676f649dcb7e7c469b3148bc3 (127.13.195.66:41405): tablet 4ad07207c150494aadc05df1cf95c6ae (table test-table [id=97cfd1f3a0cd47088c2454e18bea37e9]) successfully deleted
W20260619 20:44:50.926551 20845 catalog_manager.cc:4584] TS 875aa58d46f948a7ab7216a45fb133dc (127.13.195.65:41687): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ad07207c150494aadc05df1cf95c6ae: Network error: Client connection negotiation failed: client connection to 127.13.195.65:41687: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:44:50.926612 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 21233
I20260619 20:44:50.933707 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 21369
I20260619 20:44:50.937901 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 20726
I20260619 20:44:50.942678 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 20914
2026-06-19T20:44:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6663 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:50 dist-test-slave-n2td krb5kdc[21631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:50 dist-test-slave-n2td krb5kdc[21631](info): set up 2 sockets
Jun 19 20:44:50 dist-test-slave-n2td krb5kdc[21631](info): commencing operation
krb5kdc: starting...
W20260619 20:44:51.989897 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab.
Entry for principal oryx/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab.
I20260619 20:44:52.121296 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32853
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab
--principal=oryx/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:32853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:52.201452 21644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:52.201661 21644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:52.201684 21644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:52.202931 21644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:52.202975 21644 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:44:52.202988 21644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:52.203001 21644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:52.203058 21644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:52.203075 21644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:52.204631 21644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:32853
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.126.keytab
--principal=oryx/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:52.204905 21644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:52.205163 21644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:52.209959 21652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:52.210000 21650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:52.209959 21649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:52.210173 21644 server_base.cc:1034] running on GCE node
I20260619 20:44:52.210546 21644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:52.210850 21644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:52.212023 21644 hybrid_clock.cc:648] HybridClock initialized: now 1781901892212009 us; error 32 us; skew 500 ppm
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:52.214630 21644 init.cc:377] Logged in from keytab as oryx/127.13.195.126@KRBTEST.COM (short username oryx)
I20260619 20:44:52.215061 21644 webserver.cc:460] Webserver started at http://127.13.195.126:37581/ using document root <none> and password file <none>
I20260619 20:44:52.215303 21644 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:52.215397 21644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:52.215551 21644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:52.216617 21644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "880c8ac1a6e84254bbb182cdf5b557da"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "d015a76d8a38f1acb201944a07e8f7da"
server_key_iv: "c5177788f3e96eee081c1d94abed5c40"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.216976 21644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "880c8ac1a6e84254bbb182cdf5b557da"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "d015a76d8a38f1acb201944a07e8f7da"
server_key_iv: "c5177788f3e96eee081c1d94abed5c40"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.218698 21644 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 20:44:52.219439 21659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.219632 21644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.219734 21644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "880c8ac1a6e84254bbb182cdf5b557da"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "d015a76d8a38f1acb201944a07e8f7da"
server_key_iv: "c5177788f3e96eee081c1d94abed5c40"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.219812 21644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:52.299464 21644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:52.301963 21644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:52.302147 21644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:52.306737 21644 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:32853
I20260619 20:44:52.306852 21721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:32853 every 8 connection(s)
I20260619 20:44:52.307193 21644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:52.307886 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:52.310192 21722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Bootstrap starting.
I20260619 20:44:52.310866 21722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:52.311170 21722 log.cc:824] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:52.311916 21722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: No bootstrap required, opened a new log
I20260619 20:44:52.314117 21722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c8ac1a6e84254bbb182cdf5b557da" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32853 } }
I20260619 20:44:52.314284 21722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:52.314335 21722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880c8ac1a6e84254bbb182cdf5b557da, State: Initialized, Role: FOLLOWER
I20260619 20:44:52.314530 21722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c8ac1a6e84254bbb182cdf5b557da" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32853 } }
I20260619 20:44:52.314617 21722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:52.314664 21722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:52.314715 21722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:52.315255 21722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c8ac1a6e84254bbb182cdf5b557da" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32853 } }
I20260619 20:44:52.315375 21722 leader_election.cc:302] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 880c8ac1a6e84254bbb182cdf5b557da; no voters:
I20260619 20:44:52.315560 21722 leader_election.cc:288] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:52.315593 21725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:52.315781 21725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [term 1 LEADER]: Becoming Leader. State: Replica: 880c8ac1a6e84254bbb182cdf5b557da, State: Running, Role: LEADER
I20260619 20:44:52.315829 21722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:52.315871 21725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c8ac1a6e84254bbb182cdf5b557da" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32853 } }
I20260619 20:44:52.316237 21727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 880c8ac1a6e84254bbb182cdf5b557da. Latest consensus state: current_term: 1 leader_uuid: "880c8ac1a6e84254bbb182cdf5b557da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c8ac1a6e84254bbb182cdf5b557da" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32853 } } }
I20260619 20:44:52.316342 21727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:52.316239 21726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "880c8ac1a6e84254bbb182cdf5b557da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c8ac1a6e84254bbb182cdf5b557da" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32853 } } }
I20260619 20:44:52.316466 21726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:52.316567 21732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:52.317166 21732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:52.317190 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 21644
I20260619 20:44:52.317301 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:52.317499 14093 external_mini_cluster.cc:1428] Setting key fa3f8d47a012db86982bbe602dc2ddf0
I20260619 20:44:52.319032 21732 catalog_manager.cc:1347] Generated new cluster ID: 8c7b9568c647498e88f1d7a92a78244a
I20260619 20:44:52.319087 21732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:52.329118 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.317955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.13.195.126:32853)
0619 20:44:52.318109 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.318117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.318143 (+ 26us) server_negotiation.cc:407] Connection header received
0619 20:44:52.318263 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.318268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.318310 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.318483 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.319210 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.320574 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.321694 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.322340 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.322460 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.325352 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.325356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.325357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.325369 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.327400 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.327797 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.327800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.327800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.327916 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.328137 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.328138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.328139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.328480 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.328643 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.328726 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.328868 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260619 20:44:52.365751 21732 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:52.366545 21732 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:52.375556 21732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Generated new TSK 0
I20260619 20:44:52.375757 21732 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab.
Entry for principal oryx/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab.
I20260619 20:44:52.405493 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32853
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab
--principal=oryx/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:52.487777 21748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:52.487972 21748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:52.487995 21748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:52.489154 21748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:52.489195 21748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:52.489215 21748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:52.489269 21748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:52.490815 21748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab
--principal=oryx/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:32853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:52.491067 21748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:52.491293 21748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:44:52.496644 21748 server_base.cc:1034] running on GCE node
W20260619 20:44:52.496603 21753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:52.496598 21754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:52.496614 21756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:52.497099 21748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:52.497340 21748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:52.498485 21748 hybrid_clock.cc:648] HybridClock initialized: now 1781901892498460 us; error 40 us; skew 500 ppm
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:52.501055 21748 init.cc:377] Logged in from keytab as oryx/127.13.195.65@KRBTEST.COM (short username oryx)
I20260619 20:44:52.501485 21748 webserver.cc:460] Webserver started at http://127.13.195.65:34493/ using document root <none> and password file <none>
I20260619 20:44:52.501746 21748 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:52.501814 21748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:52.501962 21748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:52.503082 21748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "4f5e1d279f584e8280e2246d18032763"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "e9dadf7ba09cba91d4b432fed5189e9c"
server_key_iv: "c19f7800b7f76f58622a87f280965216"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.503435 21748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "4f5e1d279f584e8280e2246d18032763"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "e9dadf7ba09cba91d4b432fed5189e9c"
server_key_iv: "c19f7800b7f76f58622a87f280965216"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.504791 21748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:52.505517 21763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.505687 21748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:52.505782 21748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "4f5e1d279f584e8280e2246d18032763"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "e9dadf7ba09cba91d4b432fed5189e9c"
server_key_iv: "c19f7800b7f76f58622a87f280965216"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.505867 21748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:52.535183 21748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:52.537082 21748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:52.537245 21748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:52.537742 21748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:52.537804 21748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.537853 21748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:52.537876 21748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.544697 21748 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:38443
I20260619 20:44:52.544782 21878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:38443 every 8 connection(s)
I20260619 20:44:52.545166 21748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:52.548837 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.538398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:47013 (local address 127.13.195.126:32853)
0619 20:44:52.538476 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.538478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.538675 (+ 197us) server_negotiation.cc:407] Connection header received
0619 20:44:52.539284 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.539286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.539299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.539353 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.540302 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.541191 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.542908 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.543139 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.543197 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.545871 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.545873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.545873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.545881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.547373 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.547820 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.547821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.547822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.547887 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.548169 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.548170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.548171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.548297 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.548376 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.548656 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.548745 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 20:44:52.549028 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.538373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32853 (local address 127.13.195.65:47013)
0619 20:44:52.538553 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.538591 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.539158 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.539457 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.539463 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.539886 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:52.540189 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.540197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.541288 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.541290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.542814 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.542817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.543225 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.543227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.543386 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.543973 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:52.543984 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:52.545764 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:52.547480 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.547487 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.547489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.547728 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.547954 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.547955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.547956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.548088 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.548367 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:52.548374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:52.548521 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.548631 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.548789 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260619 20:44:52.549533 21879 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32853
I20260619 20:44:52.549631 21879 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:52.549805 21879 heartbeater.cc:511] Master 127.13.195.126:32853 requested a full tablet report, sending...
I20260619 20:44:52.550364 21675 ts_manager.cc:194] Registered new tserver with Master: 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443)
I20260619 20:44:52.551084 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 21748
I20260619 20:44:52.551189 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:52.551277 21675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:47013
I20260619 20:44:52.551424 14093 external_mini_cluster.cc:1428] Setting key c3f0f5518ab690bbfe9e18d4ff32b4b6
I20260619 20:44:52.556854 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.550582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32853 (local address 127.13.195.65:33167)
0619 20:44:52.550703 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.550713 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.550794 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.550922 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.550923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.551100 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:52.551179 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.551181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.552212 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.552213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.553048 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.553050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.553370 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.553371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.553442 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.553889 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:52.553893 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:52.554153 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:52.555870 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.555871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.555871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.556099 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.556312 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.556313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.556313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.556375 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.556638 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:52.556638 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:52.556671 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.556720 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.556792 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260619 20:44:52.556960 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.550622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:33167 (local address 127.13.195.126:32853)
0619 20:44:52.550661 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.550664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.550780 (+ 116us) server_negotiation.cc:407] Connection header received
0619 20:44:52.550851 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.550852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.550862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.550917 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.551273 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.552106 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.553125 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.553298 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.553352 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.554238 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.554240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.554240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.554248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.555784 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.556174 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.556176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.556176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.556240 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.556436 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.556437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.556437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.556571 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.556645 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.556748 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.556830 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":15}
WARNING: no policy specified for oryx/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab.
Entry for principal oryx/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab.
I20260619 20:44:52.577822 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32853
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab
--principal=oryx/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:52.659284 21888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:52.659512 21888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:52.659538 21888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:52.660739 21888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:52.660790 21888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:52.660811 21888 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:52.660866 21888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:52.662405 21888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab
--principal=oryx/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:32853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:52.662703 21888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:52.662950 21888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:52.668052 21894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:52.668177 21888 server_base.cc:1034] running on GCE node
W20260619 20:44:52.668026 21893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:52.668028 21896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:52.668510 21888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:52.668805 21888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:52.669962 21888 hybrid_clock.cc:648] HybridClock initialized: now 1781901892669946 us; error 48 us; skew 500 ppm
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:52.672595 21888 init.cc:377] Logged in from keytab as oryx/127.13.195.66@KRBTEST.COM (short username oryx)
I20260619 20:44:52.672971 21888 webserver.cc:460] Webserver started at http://127.13.195.66:34943/ using document root <none> and password file <none>
I20260619 20:44:52.673161 21888 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:52.673202 21888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:52.673295 21888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:52.674294 21888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "5122bb8404ad4ac5b33d80b043a6316d"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "53e3cb63a580a613f1ec2be9eea4e1f1"
server_key_iv: "4830b270307f2ab3a3a3b90c4fa8ad2f"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.674669 21888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "5122bb8404ad4ac5b33d80b043a6316d"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "53e3cb63a580a613f1ec2be9eea4e1f1"
server_key_iv: "4830b270307f2ab3a3a3b90c4fa8ad2f"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.675902 21888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:52.676554 21903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.676723 21888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:52.676821 21888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "5122bb8404ad4ac5b33d80b043a6316d"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "53e3cb63a580a613f1ec2be9eea4e1f1"
server_key_iv: "4830b270307f2ab3a3a3b90c4fa8ad2f"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.676901 21888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:52.693842 21888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:52.695863 21888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:52.696050 21888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:52.696573 21888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:52.696638 21888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.696686 21888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:52.696718 21888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.704157 21888 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:41857
I20260619 20:44:52.704221 22018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:41857 every 8 connection(s)
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:52.704582 21888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:52.707296 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.697057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:47193 (local address 127.13.195.126:32853)
0619 20:44:52.697138 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.697140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.697419 (+ 279us) server_negotiation.cc:407] Connection header received
0619 20:44:52.698007 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.698009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.698022 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.698119 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.699128 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.699961 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.701784 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.702155 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.702223 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.705009 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.705011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.705011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.705019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.706143 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.706563 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.706564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.706564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.706616 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.706785 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.706786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.706786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.706877 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.706923 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.707168 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.707233 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:44:52.707422 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.697026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32853 (local address 127.13.195.66:47193)
0619 20:44:52.697304 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.697333 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.697889 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.698222 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.698228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.698685 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:52.699006 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.699014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.700081 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.700082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.701692 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.701697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.702260 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.702261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.702415 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.703045 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:52.703057 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:52.704900 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:52.706213 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.706218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.706220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.706494 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.706659 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.706660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.706660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.706748 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.706929 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:52.706933 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:52.707049 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.707124 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.707240 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260619 20:44:52.707896 22019 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32853
I20260619 20:44:52.707983 22019 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:52.708192 22019 heartbeater.cc:511] Master 127.13.195.126:32853 requested a full tablet report, sending...
I20260619 20:44:52.708648 21676 ts_manager.cc:194] Registered new tserver with Master: 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857)
I20260619 20:44:52.709170 21676 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:47193
I20260619 20:44:52.712975 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 21888
I20260619 20:44:52.713106 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:52.713310 14093 external_mini_cluster.cc:1428] Setting key 79c9e1498faa8c39dbc601c3c48ecbdb
I20260619 20:44:52.714890 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.709085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:60631 (local address 127.13.195.126:32853)
0619 20:44:52.709128 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.709130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.709201 (+ 71us) server_negotiation.cc:407] Connection header received
0619 20:44:52.709293 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.709295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.709306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.709369 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.709880 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.710375 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.711110 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.711290 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.711347 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.712145 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.712146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.712146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.712151 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.713539 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.714049 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.714050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.714050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.714108 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.714304 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.714305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.714305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.714495 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.714576 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.714724 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.714800 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:44:52.714905 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.708959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32853 (local address 127.13.195.66:60631)
0619 20:44:52.709130 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.709142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.709216 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.709385 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.709386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.709724 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:52.709788 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.709790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.710535 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.710536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.711021 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.711022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.711360 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.711360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.711408 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.711799 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:52.711803 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:52.712073 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:52.713809 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.713810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.713810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.713966 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.714174 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.714174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.714175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.714231 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.714566 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:52.714567 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:52.714611 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.714700 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.714802 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab.
Entry for principal oryx/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab.
I20260619 20:44:52.737854 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32853
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab
--principal=oryx/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:52.819728 22028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:52.819973 22028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:52.819999 22028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:52.821112 22028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:52.821153 22028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:52.821177 22028 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:44:52.821197 22028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:52.822872 22028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab
--principal=oryx/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.195.126:32853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:52.823114 22028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:52.823383 22028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:52.828449 22033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:52.828505 22028 server_base.cc:1034] running on GCE node
W20260619 20:44:52.828451 22036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:52.828538 22034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:52.828871 22028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:52.829180 22028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:52.830350 22028 hybrid_clock.cc:648] HybridClock initialized: now 1781901892830335 us; error 32 us; skew 500 ppm
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:52.832844 22028 init.cc:377] Logged in from keytab as oryx/127.13.195.67@KRBTEST.COM (short username oryx)
I20260619 20:44:52.833323 22028 webserver.cc:460] Webserver started at http://127.13.195.67:34415/ using document root <none> and password file <none>
I20260619 20:44:52.833505 22028 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:52.833554 22028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:52.833647 22028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:52.834766 22028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "92ff8c92ad534e0f85971b7571c037f3"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "75ae92713d438fbe1fae14174c94a1b8"
server_key_iv: "7012a0f825574f631d30747370abce67"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.835074 22028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "92ff8c92ad534e0f85971b7571c037f3"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "75ae92713d438fbe1fae14174c94a1b8"
server_key_iv: "7012a0f825574f631d30747370abce67"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.836391 22028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:44:52.837075 22043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.837245 22028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:52.837352 22028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "92ff8c92ad534e0f85971b7571c037f3"
format_stamp: "Formatted at 2026-06-19 20:44:52 on dist-test-slave-n2td"
server_key: "75ae92713d438fbe1fae14174c94a1b8"
server_key_iv: "7012a0f825574f631d30747370abce67"
server_key_version: "encryptionkey@0"
I20260619 20:44:52.837426 22028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:52.852383 22028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:52.854255 22028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:52.854454 22028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:52.855084 22028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:52.855134 22028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.855180 22028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:52.855229 22028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:52.862331 22028 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:40595
I20260619 20:44:52.862459 22158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:40595 every 8 connection(s)
I20260619 20:44:52.862763 22028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:52.863408 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 22028
I20260619 20:44:52.863536 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:52.863907 14093 external_mini_cluster.cc:1428] Setting key 5f84b85b1769a59435843e3d66be8b92
I20260619 20:44:52.865805 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.855480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:57885 (local address 127.13.195.126:32853)
0619 20:44:52.855554 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.855556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.855756 (+ 200us) server_negotiation.cc:407] Connection header received
0619 20:44:52.856227 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.856228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.856243 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.856325 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.856997 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.857827 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.859617 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.859895 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.859988 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.862770 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.862772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.862772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.862781 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.864379 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.864805 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.864806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.864806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.864871 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.865138 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.865139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.865139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.865272 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.865350 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.865631 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.865716 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:44:52.866007 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.855473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32853 (local address 127.13.195.67:57885)
0619 20:44:52.855620 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.855661 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.856131 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.856388 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.856391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.856646 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:52.856893 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.856900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.857935 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.857936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.859508 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.859513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.860011 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.860013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.860140 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.861113 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:52.861121 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:52.862669 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:52.864475 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.864481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.864484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.864711 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.864937 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.864938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.864938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.865057 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.865339 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:52.865345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:52.865490 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.865606 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.865770 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260619 20:44:52.866488 22159 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32853
I20260619 20:44:52.866575 22159 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:52.866766 22159 heartbeater.cc:511] Master 127.13.195.126:32853 requested a full tablet report, sending...
I20260619 20:44:52.867175 21675 ts_manager.cc:194] Registered new tserver with Master: 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595)
I20260619 20:44:52.867681 21675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:57885
I20260619 20:44:52.872529 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.867583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:47063 (local address 127.13.195.126:32853)
0619 20:44:52.867813 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.867815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.867823 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:52.867841 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.867842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.867851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.867899 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.868273 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.868737 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.869372 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.869549 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.869595 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.870344 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.870345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.870345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.870350 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.871750 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.872016 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.872017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.872017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.872063 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.872205 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.872205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.872206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.872287 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.872332 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.872412 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.872463 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260619 20:44:52.872535 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.867499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32853 (local address 127.13.195.67:47063)
0619 20:44:52.867629 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.867641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.867694 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.867904 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.867905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.868124 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:52.868195 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.868197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.868827 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.868828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.869307 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.869308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.869608 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.869609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.869656 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.870009 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:52.870013 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:52.870273 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:52.871813 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.871814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.871814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.871958 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.872114 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:52.872115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:52.872115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:52.872162 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:52.872321 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:52.872321 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:52.872353 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.872397 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.872466 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260619 20:44:52.875573 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:44:52.895830 21744 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:52.890398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.13.195.126:32853)
0619 20:44:52.890473 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.890476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.890486 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:52.890552 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.890552 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.890570 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.890636 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.891206 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.891691 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.892450 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.892648 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.892732 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.895732 (+ 3000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33396: BlockingRecv error: recv got EOF from 127.0.0.1:33396 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 20:44:52 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:52.903069 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.896583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.13.195.126:32853)
0619 20:44:52.896635 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.896636 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.896725 (+ 89us) server_negotiation.cc:407] Connection header received
0619 20:44:52.896762 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.896763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.896771 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.896824 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:52.897247 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.897696 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.898319 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.898571 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.898626 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.900630 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:52.900631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:52.900632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:52.900639 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:52.902151 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.902474 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.902475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.902476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.902533 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:52.902687 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:52.902687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:52.902688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:52.902784 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:52.902853 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.902939 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.902995 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:44:52.904577 21675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:52.906176 21675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:52.912443 22185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.908403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38443 (local address 127.0.0.1:35740)
0619 20:44:52.908780 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.908790 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.908836 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.909212 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.909213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.909215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:52.909259 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.909260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.910518 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.910520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.911266 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.911268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.912142 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.912143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.912221 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.912236 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.912313 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.912365 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":84,"threads_started":1}
I20260619 20:44:52.912889 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.908511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.13.195.65:38443)
0619 20:44:52.908999 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.909003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.909013 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:52.909049 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.909051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.909068 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.909142 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:52.909333 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.910416 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.911370 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.912056 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.912131 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.912206 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.912748 (+ 542us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.912792 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":81,"threads_started":1}
I20260619 20:44:52.912892 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.908573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40595 (local address 127.0.0.1:52202)
0619 20:44:52.908794 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.908805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.908880 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.909217 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.909218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.909220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:52.909268 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.909270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.910195 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.910196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.911780 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.911782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.912646 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.912647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.912691 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.912694 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.912764 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.912806 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":32,"thread_start_us":64,"threads_started":1}
I20260619 20:44:52.912904 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.908781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.13.195.67:40595)
0619 20:44:52.909016 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.909020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.909030 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:52.909066 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.909068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.909082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.909157 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:52.909346 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.910110 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.911864 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.912485 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.912561 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.912618 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.912777 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.912811 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":92,"threads_started":1}
I20260619 20:44:52.913123 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.908391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:41857 (local address 127.0.0.1:57666)
0619 20:44:52.908569 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.908613 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.908679 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.908930 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.908933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.908935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:52.909060 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.909063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.910390 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.910391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.911532 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.911534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.912635 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.912636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.912708 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.912712 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.912756 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.912793 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":7,"thread_start_us":47,"threads_started":1}
I20260619 20:44:52.913247 22183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.908496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.13.195.66:41857)
0619 20:44:52.908684 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.908690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.908711 (+ 21us) server_negotiation.cc:407] Connection header received
0619 20:44:52.908792 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.908794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.908811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.908930 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:52.909152 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.910295 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.911617 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.912366 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.912434 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.912506 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.912853 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.912891 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260619 20:44:52.913594 22093 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8eae68f1af4e3caa7e223eee692dde (DEFAULT_TABLE table=test-table [id=750ae882e86c4d65bb38da84d07f11d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:52.913817 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8eae68f1af4e3caa7e223eee692dde (DEFAULT_TABLE table=test-table [id=750ae882e86c4d65bb38da84d07f11d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:52.913949 22093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8eae68f1af4e3caa7e223eee692dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:52.914126 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8eae68f1af4e3caa7e223eee692dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:52.915040 21813 tablet_service.cc:1467] Processing CreateTablet for tablet 0d8eae68f1af4e3caa7e223eee692dde (DEFAULT_TABLE table=test-table [id=750ae882e86c4d65bb38da84d07f11d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:52.915401 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d8eae68f1af4e3caa7e223eee692dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:52.917284 22187 tablet_bootstrap.cc:492] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Bootstrap starting.
I20260619 20:44:52.917956 22187 tablet_bootstrap.cc:654] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:52.917963 22188 tablet_bootstrap.cc:492] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Bootstrap starting.
I20260619 20:44:52.918390 22187 log.cc:824] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:52.918633 22188 tablet_bootstrap.cc:654] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:52.918924 22188 log.cc:824] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:52.919256 22191 tablet_bootstrap.cc:492] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Bootstrap starting.
I20260619 20:44:52.919730 22187 tablet_bootstrap.cc:492] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: No bootstrap required, opened a new log
I20260619 20:44:52.919730 22188 tablet_bootstrap.cc:492] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: No bootstrap required, opened a new log
I20260619 20:44:52.919803 22187 ts_tablet_manager.cc:1397] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:44:52.919803 22188 ts_tablet_manager.cc:1397] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:52.919900 22191 tablet_bootstrap.cc:654] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:52.920169 22191 log.cc:824] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:52.920854 22191 tablet_bootstrap.cc:492] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: No bootstrap required, opened a new log
I20260619 20:44:52.920941 22191 ts_tablet_manager.cc:1397] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:52.922206 22187 raft_consensus.cc:348] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.922418 22187 raft_consensus.cc:374] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:52.922477 22187 raft_consensus.cc:729] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Initialized, Role: FOLLOWER
I20260619 20:44:52.922391 22188 raft_consensus.cc:348] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.922561 22188 raft_consensus.cc:374] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:52.922603 22188 raft_consensus.cc:729] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5122bb8404ad4ac5b33d80b043a6316d, State: Initialized, Role: FOLLOWER
I20260619 20:44:52.922600 22187 consensus_queue.cc:260] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.922497 22191 raft_consensus.cc:348] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.922688 22191 raft_consensus.cc:374] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:52.922729 22191 raft_consensus.cc:729] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5e1d279f584e8280e2246d18032763, State: Initialized, Role: FOLLOWER
I20260619 20:44:52.922729 22188 consensus_queue.cc:260] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.922858 22191 consensus_queue.cc:260] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.922902 22187 ts_tablet_manager.cc:1428] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 20:44:52.923069 22188 ts_tablet_manager.cc:1428] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 20:44:52.923074 22159 heartbeater.cc:503] Master 127.13.195.126:32853 was elected leader, sending a full tablet report...
I20260619 20:44:52.923105 22191 ts_tablet_manager.cc:1428] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:44:52.923200 22019 heartbeater.cc:503] Master 127.13.195.126:32853 was elected leader, sending a full tablet report...
I20260619 20:44:52.923228 21879 heartbeater.cc:503] Master 127.13.195.126:32853 was elected leader, sending a full tablet report...
I20260619 20:44:52.929899 22193 raft_consensus.cc:482] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:52.929986 22193 raft_consensus.cc:504] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.930356 22193 leader_election.cc:288] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443), 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857)
I20260619 20:44:52.934217 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.930648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:42143 (local address 127.13.195.65:38443)
0619 20:44:52.930698 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.930700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.930708 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:52.930735 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.930736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.930745 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.930795 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:52.931028 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.931774 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.933289 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.933834 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.933918 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.933927 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.934114 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.934151 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:44:52.934216 22183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.930598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:57989 (local address 127.13.195.66:41857)
0619 20:44:52.930638 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.930640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.930865 (+ 225us) server_negotiation.cc:407] Connection header received
0619 20:44:52.930919 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.930920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.930930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.930984 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:52.931128 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.931807 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.933288 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.933855 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.933922 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.933929 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.934114 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.934150 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:44:52.934229 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.930610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:41857 (local address 127.13.195.67:57989)
0619 20:44:52.930803 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.930813 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.930876 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.931000 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.931001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.931002 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:52.931063 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.931064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.931875 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.931875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.933184 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.933188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.933930 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.933931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.934021 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.934025 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.934085 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.934127 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":14,"thread_start_us":149,"threads_started":1}
I20260619 20:44:52.934233 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.930568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38443 (local address 127.13.195.67:42143)
0619 20:44:52.930621 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:52.930633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:52.930675 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:52.930796 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:52.930797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:52.930798 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:52.930868 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.930870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.931869 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.931871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.933184 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:52.933188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.933930 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.933931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:52.934024 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.934025 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:52.934096 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:44:52.934127 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":5}
I20260619 20:44:52.934612 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8eae68f1af4e3caa7e223eee692dde" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d" is_pre_election: true
I20260619 20:44:52.934769 21973 raft_consensus.cc:2454] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 0.
I20260619 20:44:52.934871 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8eae68f1af4e3caa7e223eee692dde" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5e1d279f584e8280e2246d18032763" is_pre_election: true
I20260619 20:44:52.934993 22044 leader_election.cc:302] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5122bb8404ad4ac5b33d80b043a6316d, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:52.935021 21833 raft_consensus.cc:2454] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 0.
I20260619 20:44:52.935110 22193 raft_consensus.cc:2781] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:52.935168 22193 raft_consensus.cc:482] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:52.935191 22193 raft_consensus.cc:3037] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:52.936228 22193 raft_consensus.cc:504] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.936367 22193 leader_election.cc:288] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443), 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857)
I20260619 20:44:52.936527 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8eae68f1af4e3caa7e223eee692dde" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5e1d279f584e8280e2246d18032763"
I20260619 20:44:52.936524 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d8eae68f1af4e3caa7e223eee692dde" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d"
I20260619 20:44:52.936590 21833 raft_consensus.cc:3037] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:52.936614 21973 raft_consensus.cc:3037] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:52.937263 21973 raft_consensus.cc:2454] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 1.
I20260619 20:44:52.937309 21833 raft_consensus.cc:2454] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 1.
I20260619 20:44:52.937467 22046 leader_election.cc:302] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:52.937547 22193 raft_consensus.cc:2781] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:52.937680 22193 raft_consensus.cc:686] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 1 LEADER]: Becoming Leader. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Running, Role: LEADER
I20260619 20:44:52.937780 22193 consensus_queue.cc:237] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } }
I20260619 20:44:52.938603 21675 catalog_manager.cc:5495] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:52.947641 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:52.945323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.13.195.67:40595)
0619 20:44:52.945420 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:52.945421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:52.945429 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:52.945454 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:52.945455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:52.945471 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:52.945528 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:52.945656 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.946106 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.946911 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:52.947091 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:52.947143 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:52.947277 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:52.947369 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:52.947429 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:52.947528 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 20:44:52.947544 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 20:44:52.949558 21833 raft_consensus.cc:1261] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Refusing update from remote peer 92ff8c92ad534e0f85971b7571c037f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:52.949523 21973 raft_consensus.cc:1261] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Refusing update from remote peer 92ff8c92ad534e0f85971b7571c037f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:52.949900 22197 consensus_queue.cc:1035] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:52.950030 22193 consensus_queue.cc:1035] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:52.952443 22200 mvcc.cc:205] Tried to move back new op lower bound from 7298670153518825472 to 7298670153473794048. Current Snapshot: MvccSnapshot[applied={T|T < 7298670153518825472}]
I20260619 20:44:52.953066 22201 mvcc.cc:205] Tried to move back new op lower bound from 7298670153518825472 to 7298670153473794048. Current Snapshot: MvccSnapshot[applied={T|T < 7298670153518825472}]
I20260619 20:44:52.954984 21675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:52.955087 21675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:52.955950 21675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Sending DeleteTablet for 3 replicas of tablet 0d8eae68f1af4e3caa7e223eee692dde
I20260619 20:44:52.956293 21813 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d8eae68f1af4e3caa7e223eee692dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:52 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:35740
I20260619 20:44:52.956293 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d8eae68f1af4e3caa7e223eee692dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:52 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:57666
I20260619 20:44:52.956789 22210 tablet_replica.cc:330] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: stopping tablet replica
I20260619 20:44:52.956773 22093 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d8eae68f1af4e3caa7e223eee692dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:52 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:52202
I20260619 20:44:52.956804 21675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 20:44:52.956900 22210 raft_consensus.cc:2229] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 20:44:52.956948 21675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:52.956992 22211 tablet_replica.cc:330] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: stopping tablet replica
I20260619 20:44:52.957021 22210 raft_consensus.cc:2258] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:52.957063 22211 raft_consensus.cc:2229] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:52.957149 22211 raft_consensus.cc:2258] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:52.957369 22209 tablet_replica.cc:330] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: stopping tablet replica
I20260619 20:44:52.957464 22209 raft_consensus.cc:2229] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:52.957563 22209 raft_consensus.cc:2258] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:52.957576 22211 ts_tablet_manager.cc:1905] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:52.958592 21813 tablet_service.cc:1467] Processing CreateTablet for tablet 76688e4bd00a4762af9061e5f3773cbd (DEFAULT_TABLE table=test-table [id=af340dd56202431c8fb03e6a16a86632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:52.958675 22093 tablet_service.cc:1467] Processing CreateTablet for tablet 76688e4bd00a4762af9061e5f3773cbd (DEFAULT_TABLE table=test-table [id=af340dd56202431c8fb03e6a16a86632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:52.958721 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76688e4bd00a4762af9061e5f3773cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:52.958781 22093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76688e4bd00a4762af9061e5f3773cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:52.959012 22210 ts_tablet_manager.cc:1905] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:52.960101 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 76688e4bd00a4762af9061e5f3773cbd (DEFAULT_TABLE table=test-table [id=af340dd56202431c8fb03e6a16a86632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:52.960188 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76688e4bd00a4762af9061e5f3773cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:52.960579 22210 ts_tablet_manager.cc:1918] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:52.960654 22210 log.cc:1196] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/0d8eae68f1af4e3caa7e223eee692dde
I20260619 20:44:52.960934 22210 ts_tablet_manager.cc:1939] T 0d8eae68f1af4e3caa7e223eee692dde P 5122bb8404ad4ac5b33d80b043a6316d: Deleting consensus metadata
I20260619 20:44:52.961356 21660 catalog_manager.cc:4841] TS 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857): tablet 0d8eae68f1af4e3caa7e223eee692dde (table test-table [id=750ae882e86c4d65bb38da84d07f11d9]) successfully deleted
I20260619 20:44:52.961431 22187 tablet_bootstrap.cc:492] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Bootstrap starting.
I20260619 20:44:52.961862 22187 tablet_bootstrap.cc:654] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:52.961961 22211 ts_tablet_manager.cc:1918] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:52.962033 22211 log.cc:1196] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/0d8eae68f1af4e3caa7e223eee692dde
I20260619 20:44:52.962224 22191 tablet_bootstrap.cc:492] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Bootstrap starting.
I20260619 20:44:52.962294 22211 ts_tablet_manager.cc:1939] T 0d8eae68f1af4e3caa7e223eee692dde P 92ff8c92ad534e0f85971b7571c037f3: Deleting consensus metadata
I20260619 20:44:52.962632 22187 tablet_bootstrap.cc:492] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: No bootstrap required, opened a new log
I20260619 20:44:52.962670 22191 tablet_bootstrap.cc:654] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:52.962680 22187 ts_tablet_manager.cc:1397] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 20:44:52.962818 22187 raft_consensus.cc:348] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:52.962882 22187 raft_consensus.cc:374] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:52.962903 22187 raft_consensus.cc:729] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Initialized, Role: FOLLOWER
I20260619 20:44:52.962946 22187 consensus_queue.cc:260] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:52.962970 22188 tablet_bootstrap.cc:492] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: Bootstrap starting.
I20260619 20:44:52.963035 22187 ts_tablet_manager.cc:1428] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:52.963233 22191 tablet_bootstrap.cc:492] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: No bootstrap required, opened a new log
I20260619 20:44:52.963284 22191 ts_tablet_manager.cc:1397] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:52.963371 22188 tablet_bootstrap.cc:654] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:52.963481 22191 raft_consensus.cc:348] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:52.963575 22191 raft_consensus.cc:374] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:52.963598 22191 raft_consensus.cc:729] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5e1d279f584e8280e2246d18032763, State: Initialized, Role: FOLLOWER
I20260619 20:44:52.963650 22191 consensus_queue.cc:260] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:52.963753 22191 ts_tablet_manager.cc:1428] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:52.964032 21662 catalog_manager.cc:4841] TS 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595): tablet 0d8eae68f1af4e3caa7e223eee692dde (table test-table [id=750ae882e86c4d65bb38da84d07f11d9]) successfully deleted
I20260619 20:44:52.964488 22209 ts_tablet_manager.cc:1905] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:52.965740 22209 ts_tablet_manager.cc:1918] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:52.965797 22209 log.cc:1196] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/0d8eae68f1af4e3caa7e223eee692dde
I20260619 20:44:52.965974 22209 ts_tablet_manager.cc:1939] T 0d8eae68f1af4e3caa7e223eee692dde P 4f5e1d279f584e8280e2246d18032763: Deleting consensus metadata
I20260619 20:44:52.966369 21662 catalog_manager.cc:4841] TS 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443): tablet 0d8eae68f1af4e3caa7e223eee692dde (table test-table [id=750ae882e86c4d65bb38da84d07f11d9]) successfully deleted
I20260619 20:44:52.970827 22188 tablet_bootstrap.cc:492] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: No bootstrap required, opened a new log
I20260619 20:44:52.970875 22188 ts_tablet_manager.cc:1397] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 20:44:52.971041 22188 raft_consensus.cc:348] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:52.971101 22188 raft_consensus.cc:374] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:52.971115 22188 raft_consensus.cc:729] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5122bb8404ad4ac5b33d80b043a6316d, State: Initialized, Role: FOLLOWER
I20260619 20:44:52.971174 22188 consensus_queue.cc:260] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:52.971266 22188 ts_tablet_manager.cc:1428] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.031502 22195 raft_consensus.cc:482] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:53.031595 22195 raft_consensus.cc:504] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.031955 22195 leader_election.cc:288] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857), 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595)
I20260619 20:44:53.035374 22183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.032233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:57613 (local address 127.13.195.66:41857)
0619 20:44:53.032277 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.032278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.032286 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:44:53.032314 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.032314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.032323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.032374 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:53.032540 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.033265 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.034373 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.034979 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.035057 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.035066 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.035265 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.035303 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:44:53.035389 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.032119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:41857 (local address 127.13.195.65:57613)
0619 20:44:53.032162 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.032176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:53.032227 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:53.032378 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:53.032379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:53.032381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:53.032454 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.032456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.033363 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.033364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.034268 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.034270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.035079 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.035080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.035174 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.035179 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:44:53.035251 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:44:53.035295 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 20:44:53.035696 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76688e4bd00a4762af9061e5f3773cbd" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d" is_pre_election: true
I20260619 20:44:53.035804 21973 raft_consensus.cc:2454] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 0.
I20260619 20:44:53.036121 21764 leader_election.cc:302] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 5122bb8404ad4ac5b33d80b043a6316d; no voters:
I20260619 20:44:53.036255 22195 raft_consensus.cc:2781] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:53.036293 22195 raft_consensus.cc:482] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:53.036312 22195 raft_consensus.cc:3037] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.036563 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.032334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:50445 (local address 127.13.195.67:40595)
0619 20:44:53.032386 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.032388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.033366 (+ 978us) server_negotiation.cc:407] Connection header received
0619 20:44:53.033394 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.033395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.033406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.033466 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:44:53.033713 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.034402 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.035643 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.036198 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.036296 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.036306 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.036458 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.036496 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:44:53.036579 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.032158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40595 (local address 127.13.195.65:50445)
0619 20:44:53.032441 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.032451 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:53.032502 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:53.033479 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:53.033480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:53.033482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:44:53.033625 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.033627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.034515 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.034516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.035553 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.035555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.036301 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.036302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.036381 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.036384 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:44:53.036438 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:44:53.036479 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":73,"thread_start_us":78,"threads_started":1}
I20260619 20:44:53.036916 22113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76688e4bd00a4762af9061e5f3773cbd" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ff8c92ad534e0f85971b7571c037f3" is_pre_election: true
I20260619 20:44:53.037053 22113 raft_consensus.cc:2454] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 0.
I20260619 20:44:53.037276 22195 raft_consensus.cc:504] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.037415 22195 leader_election.cc:288] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 election: Requested vote from peers 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857), 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595)
I20260619 20:44:53.037544 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76688e4bd00a4762af9061e5f3773cbd" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d"
I20260619 20:44:53.037570 22113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76688e4bd00a4762af9061e5f3773cbd" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ff8c92ad534e0f85971b7571c037f3"
I20260619 20:44:53.037640 22113 raft_consensus.cc:3037] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.037642 21973 raft_consensus.cc:3037] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.038208 21973 raft_consensus.cc:2454] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 1.
I20260619 20:44:53.038354 21764 leader_election.cc:302] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 5122bb8404ad4ac5b33d80b043a6316d; no voters:
I20260619 20:44:53.038462 22195 raft_consensus.cc:2781] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:53.038535 22113 raft_consensus.cc:2454] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 1.
I20260619 20:44:53.038606 22195 raft_consensus.cc:686] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 1 LEADER]: Becoming Leader. State: Replica: 4f5e1d279f584e8280e2246d18032763, State: Running, Role: LEADER
I20260619 20:44:53.038669 22195 consensus_queue.cc:237] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.039423 21675 catalog_manager.cc:5495] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f5e1d279f584e8280e2246d18032763 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "4f5e1d279f584e8280e2246d18032763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } health_report { overall_health: UNKNOWN } } }
W20260619 20:44:53.045885 21880 tablet.cc:2369] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901892, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.126@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:53.082413 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.075301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.13.195.126:32853)
0619 20:44:53.075391 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.075394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.075550 (+ 156us) server_negotiation.cc:407] Connection header received
0619 20:44:53.075656 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.075656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.075668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.075715 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:53.076207 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.076697 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.077395 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.077586 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.077673 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.079821 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:53.079822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:53.079823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:53.079830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:53.081358 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:53.081733 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:53.081733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:53.081734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:53.081784 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:53.081951 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:53.081951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:53.081952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:53.082056 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:53.082106 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.082230 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.082351 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":59}
I20260619 20:44:53.082401 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.075221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32853 (local address 127.0.0.1:33416)
0619 20:44:53.075434 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.075483 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:53.075573 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:53.075729 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:53.075730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:53.076063 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:53.076112 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.076115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.076787 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.076788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.077311 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.077312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.077675 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.077676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.077730 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.078249 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:53.078253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:53.079725 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:53.081450 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:53.081457 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:53.081459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:53.081661 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:53.081842 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:53.081843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:53.081843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:53.081898 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:53.082110 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:53.082116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:53.082159 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:44:53.082206 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:44:53.082309 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260619 20:44:53.084677 21675 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:33416:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:44:53.084914 21675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:53.086354 22093 tablet_service.cc:1467] Processing CreateTablet for tablet 973b85394c434cbca6e131db5d367c85 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9887d1190b2d4aa5bd764472c3cdd911]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:53.086349 21813 tablet_service.cc:1467] Processing CreateTablet for tablet 973b85394c434cbca6e131db5d367c85 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9887d1190b2d4aa5bd764472c3cdd911]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:53.086537 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 973b85394c434cbca6e131db5d367c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.086536 22093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 973b85394c434cbca6e131db5d367c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.087985 22191 tablet_bootstrap.cc:492] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: Bootstrap starting.
I20260619 20:44:53.088101 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 973b85394c434cbca6e131db5d367c85 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9887d1190b2d4aa5bd764472c3cdd911]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:44:53.088233 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 973b85394c434cbca6e131db5d367c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.088382 22191 tablet_bootstrap.cc:654] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.088951 22191 tablet_bootstrap.cc:492] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: No bootstrap required, opened a new log
I20260619 20:44:53.089013 22191 ts_tablet_manager.cc:1397] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:53.089162 22191 raft_consensus.cc:348] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.089236 22191 raft_consensus.cc:374] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.089268 22191 raft_consensus.cc:729] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5e1d279f584e8280e2246d18032763, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.089318 22191 consensus_queue.cc:260] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.089416 22191 ts_tablet_manager.cc:1428] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.089531 22188 tablet_bootstrap.cc:492] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: Bootstrap starting.
I20260619 20:44:53.089480 22195 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.089586 22195 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:53.089967 22188 tablet_bootstrap.cc:654] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.090327 22187 tablet_bootstrap.cc:492] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: Bootstrap starting.
I20260619 20:44:53.090659 22188 tablet_bootstrap.cc:492] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: No bootstrap required, opened a new log
I20260619 20:44:53.090675 22187 tablet_bootstrap.cc:654] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.090716 22188 ts_tablet_manager.cc:1397] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:53.090878 22188 raft_consensus.cc:348] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.090965 22188 raft_consensus.cc:374] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.091001 22188 raft_consensus.cc:729] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5122bb8404ad4ac5b33d80b043a6316d, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.091050 22188 consensus_queue.cc:260] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.091153 22188 ts_tablet_manager.cc:1428] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.091216 22194 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.091295 22194 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:53.091554 22187 tablet_bootstrap.cc:492] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: No bootstrap required, opened a new log
I20260619 20:44:53.091607 22187 ts_tablet_manager.cc:1397] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:53.091724 22187 raft_consensus.cc:348] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.091799 22187 raft_consensus.cc:374] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.091821 22187 raft_consensus.cc:729] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.091882 22187 consensus_queue.cc:260] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.091984 22187 ts_tablet_manager.cc:1428] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.092000 22193 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.092062 22193 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:53.096765 22193 raft_consensus.cc:482] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:53.096820 22193 raft_consensus.cc:504] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.096941 22193 leader_election.cc:288] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443), 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857)
I20260619 20:44:53.097097 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973b85394c434cbca6e131db5d367c85" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5e1d279f584e8280e2246d18032763" is_pre_election: true
I20260619 20:44:53.097189 21833 raft_consensus.cc:2454] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 0.
I20260619 20:44:53.097154 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973b85394c434cbca6e131db5d367c85" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d" is_pre_election: true
I20260619 20:44:53.097265 21973 raft_consensus.cc:2454] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 0.
I20260619 20:44:53.097347 22046 leader_election.cc:302] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:53.097442 22193 raft_consensus.cc:2781] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:53.097482 22193 raft_consensus.cc:482] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:53.097497 22193 raft_consensus.cc:3037] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.098019 22193 raft_consensus.cc:504] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.098155 22193 leader_election.cc:288] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443), 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857)
I20260619 20:44:53.098296 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973b85394c434cbca6e131db5d367c85" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5e1d279f584e8280e2246d18032763"
I20260619 20:44:53.098371 21833 raft_consensus.cc:3037] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.098312 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "973b85394c434cbca6e131db5d367c85" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d"
I20260619 20:44:53.098455 21973 raft_consensus.cc:3037] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.099016 21973 raft_consensus.cc:2454] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 1.
I20260619 20:44:53.099018 21833 raft_consensus.cc:2454] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 1.
I20260619 20:44:53.099217 22046 leader_election.cc:302] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:53.099341 22193 raft_consensus.cc:2781] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:53.099402 22193 raft_consensus.cc:686] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 LEADER]: Becoming Leader. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Running, Role: LEADER
I20260619 20:44:53.099450 22193 consensus_queue.cc:237] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.099761 22197 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: TxnStatusTablet state changed. Reason: New leader 92ff8c92ad534e0f85971b7571c037f3. Latest consensus state: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.099860 22197 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:53.100070 22228 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:44:53.100102 21675 catalog_manager.cc:5495] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } health_report { overall_health: HEALTHY } } }
I20260619 20:44:53.100195 22228 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:44:53.107002 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.104583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.13.195.67:40595)
0619 20:44:53.104745 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.104747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.104753 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:44:53.104779 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.104781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.104789 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.104845 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:53.105014 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.105487 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.106224 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.106417 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.106554 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.106743 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.106776 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.106843 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.106913 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.106934 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260619 20:44:53.107020 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.104530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40595 (local address 127.0.0.1:52222)
0619 20:44:53.104609 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.104621 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:53.104676 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:53.104871 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:53.104872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:53.104874 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:53.104937 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.104939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.105570 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.105571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.106137 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.106139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.106563 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.106565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.106635 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.106649 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.106845 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.106854 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:44:53.106905 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:44:53.106952 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 20:44:53.107838 21833 raft_consensus.cc:1261] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Refusing update from remote peer 92ff8c92ad534e0f85971b7571c037f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.107868 21973 raft_consensus.cc:1261] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Refusing update from remote peer 92ff8c92ad534e0f85971b7571c037f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.108105 22193 consensus_queue.cc:1035] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.108284 22193 consensus_queue.cc:1035] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.108652 22195 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: TxnStatusTablet state changed. Reason: New leader 92ff8c92ad534e0f85971b7571c037f3. Latest consensus state: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.108745 22195 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:53.108745 22194 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: TxnStatusTablet state changed. Reason: New leader 92ff8c92ad534e0f85971b7571c037f3. Latest consensus state: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.108829 22194 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:53.109396 22197 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.109472 22197 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:53.109561 22197 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.109607 22197 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 92ff8c92ad534e0f85971b7571c037f3: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:44:53.109732 22195 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.109809 22195 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 4f5e1d279f584e8280e2246d18032763: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:53.109861 22194 tablet_replica.cc:439] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } } }
I20260619 20:44:53.109933 22194 tablet_replica.cc:442] T 973b85394c434cbca6e131db5d367c85 P 5122bb8404ad4ac5b33d80b043a6316d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:44:53.110069 22202 mvcc.cc:205] Tried to move back new op lower bound from 7298670154168315904 to 7298670154135707648. Current Snapshot: MvccSnapshot[applied={T|T < 7298670154168315904 or (T in {7298670154168315904})}]
W20260619 20:44:53.113394 22160 tablet.cc:2369] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:53.114166 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.111546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.13.195.65:38443)
0619 20:44:53.111583 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.111585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.111629 (+ 44us) server_negotiation.cc:407] Connection header received
0619 20:44:53.111706 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.111706 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.111721 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.111776 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:53.111884 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.112330 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.113342 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.113598 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.113675 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.113834 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.113931 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.113985 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.114056 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.114071 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:44:53.119038 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.116621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:60767 (local address 127.13.195.67:40595)
0619 20:44:53.116659 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.116661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.116779 (+ 118us) server_negotiation.cc:407] Connection header received
0619 20:44:53.116860 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.116861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.116872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.116926 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:53.117101 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.117544 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.118293 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.118564 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.118645 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.118820 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.118851 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.118903 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.118953 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.118972 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:44:53.119071 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.116575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40595 (local address 127.13.195.65:60767)
0619 20:44:53.116723 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.116735 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:53.116794 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:53.116929 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:53.116930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:53.116931 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:53.117019 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.117021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.117625 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.117626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.118201 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.118203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.118647 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.118648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.118717 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.118730 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.118906 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.118908 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:44:53.118952 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:44:53.118992 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260619 20:44:53.121079 21973 raft_consensus.cc:1261] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Refusing update from remote peer 4f5e1d279f584e8280e2246d18032763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.121177 22113 raft_consensus.cc:1261] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Refusing update from remote peer 4f5e1d279f584e8280e2246d18032763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.121294 22195 consensus_queue.cc:1035] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.121424 22216 consensus_queue.cc:1035] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.129732 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.127255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:49661 (local address 127.13.195.65:38443)
0619 20:44:53.127295 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.127297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.127355 (+ 58us) server_negotiation.cc:407] Connection header received
0619 20:44:53.127426 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.127427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.127436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.127489 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:53.127611 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.128075 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.128956 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.129229 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.129298 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.129520 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.129555 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.129593 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.129665 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.129682 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:44:53.129792 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.127222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38443 (local address 127.13.195.67:49661)
0619 20:44:53.127315 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.127325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:53.127370 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:53.127491 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:53.127492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:53.127493 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:53.127544 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.127546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.128156 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.128157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.128877 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.128879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.129324 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.129326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.129408 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.129426 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.129607 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.129608 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:53.129665 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:44:53.129712 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 20:44:53.136699 21676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:53.136821 21676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:53.137522 21676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Sending DeleteTablet for 3 replicas of tablet 76688e4bd00a4762af9061e5f3773cbd
I20260619 20:44:53.137849 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 76688e4bd00a4762af9061e5f3773cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:57666
I20260619 20:44:53.137856 21813 tablet_service.cc:1514] Processing DeleteTablet for tablet 76688e4bd00a4762af9061e5f3773cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:35740
I20260619 20:44:53.137961 22209 tablet_replica.cc:330] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: stopping tablet replica
I20260619 20:44:53.137970 22210 tablet_replica.cc:330] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: stopping tablet replica
I20260619 20:44:53.138029 22209 raft_consensus.cc:2229] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:53.138031 22210 raft_consensus.cc:2229] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:53.138075 22210 raft_consensus.cc:2258] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.138119 22209 raft_consensus.cc:2258] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.138458 22209 ts_tablet_manager.cc:1905] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.138521 22210 ts_tablet_manager.cc:1905] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.140285 22093 tablet_service.cc:1514] Processing DeleteTablet for tablet 76688e4bd00a4762af9061e5f3773cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:52202
I20260619 20:44:53.140316 22209 ts_tablet_manager.cc:1918] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:53.140390 22209 log.cc:1196] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/76688e4bd00a4762af9061e5f3773cbd
I20260619 20:44:53.140411 22211 tablet_replica.cc:330] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: stopping tablet replica
I20260619 20:44:53.140460 22211 raft_consensus.cc:2229] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:53.140503 22211 raft_consensus.cc:2258] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.140669 22209 ts_tablet_manager.cc:1939] T 76688e4bd00a4762af9061e5f3773cbd P 4f5e1d279f584e8280e2246d18032763: Deleting consensus metadata
I20260619 20:44:53.140825 22211 ts_tablet_manager.cc:1905] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.142359 22211 ts_tablet_manager.cc:1918] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:53.142407 22211 log.cc:1196] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/76688e4bd00a4762af9061e5f3773cbd
I20260619 20:44:53.142652 22211 ts_tablet_manager.cc:1939] T 76688e4bd00a4762af9061e5f3773cbd P 92ff8c92ad534e0f85971b7571c037f3: Deleting consensus metadata
I20260619 20:44:53.143021 21662 catalog_manager.cc:4841] TS 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443): tablet 76688e4bd00a4762af9061e5f3773cbd (table test-table [id=af340dd56202431c8fb03e6a16a86632]) successfully deleted
I20260619 20:44:53.143250 21662 catalog_manager.cc:4841] TS 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595): tablet 76688e4bd00a4762af9061e5f3773cbd (table test-table [id=af340dd56202431c8fb03e6a16a86632]) successfully deleted
I20260619 20:44:53.144019 22210 ts_tablet_manager.cc:1918] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:53.144090 22210 log.cc:1196] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/76688e4bd00a4762af9061e5f3773cbd
I20260619 20:44:53.144325 22210 ts_tablet_manager.cc:1939] T 76688e4bd00a4762af9061e5f3773cbd P 5122bb8404ad4ac5b33d80b043a6316d: Deleting consensus metadata
I20260619 20:44:53.144651 21660 catalog_manager.cc:4841] TS 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857): tablet 76688e4bd00a4762af9061e5f3773cbd (table test-table [id=af340dd56202431c8fb03e6a16a86632]) successfully deleted
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[21631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:44:53.173228 21744 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:53.165826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.13.195.126:32853)
0619 20:44:53.165879 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.165882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.166065 (+ 183us) server_negotiation.cc:407] Connection header received
0619 20:44:53.166134 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.166135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.166150 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.166224 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:53.167066 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.168004 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.169137 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.169494 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.169574 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.173150 (+ 3576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33424: BlockingRecv error: recv got EOF from 127.0.0.1:33424 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[21631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM
I20260619 20:44:53.183794 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.174381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33436 (local address 127.13.195.126:32853)
0619 20:44:53.174419 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.174421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.174669 (+ 248us) server_negotiation.cc:407] Connection header received
0619 20:44:53.174767 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.174768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.174780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.174839 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:53.175480 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.176177 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.177319 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.177609 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.177685 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.180584 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:53.180585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:53.180586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:53.180594 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:53.182471 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:53.182883 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:53.182884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:53.182885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:53.182942 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:53.183152 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:53.183152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:53.183153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:53.183286 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:53.183378 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.183601 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.183683 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:44:53.186086 21676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:53.186261 21676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:53.187628 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 8846a61da08b4e73bb8c540326c59216 (DEFAULT_TABLE table=test-table [id=041fc72dc3bc4f388e633aaf15485894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:53.187671 21813 tablet_service.cc:1467] Processing CreateTablet for tablet 8846a61da08b4e73bb8c540326c59216 (DEFAULT_TABLE table=test-table [id=041fc72dc3bc4f388e633aaf15485894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:53.187776 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8846a61da08b4e73bb8c540326c59216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.187804 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8846a61da08b4e73bb8c540326c59216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.189160 22188 tablet_bootstrap.cc:492] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Bootstrap starting.
I20260619 20:44:53.189275 22093 tablet_service.cc:1467] Processing CreateTablet for tablet 8846a61da08b4e73bb8c540326c59216 (DEFAULT_TABLE table=test-table [id=041fc72dc3bc4f388e633aaf15485894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:53.189390 22093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8846a61da08b4e73bb8c540326c59216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.189836 22188 tablet_bootstrap.cc:654] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.190738 22191 tablet_bootstrap.cc:492] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: Bootstrap starting.
I20260619 20:44:53.191216 22191 tablet_bootstrap.cc:654] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.191495 22188 tablet_bootstrap.cc:492] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: No bootstrap required, opened a new log
I20260619 20:44:53.191547 22188 ts_tablet_manager.cc:1397] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:53.191562 22187 tablet_bootstrap.cc:492] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: Bootstrap starting.
I20260619 20:44:53.191699 22188 raft_consensus.cc:348] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.191767 22188 raft_consensus.cc:374] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.191788 22188 raft_consensus.cc:729] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5122bb8404ad4ac5b33d80b043a6316d, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.191835 22187 tablet_bootstrap.cc:654] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.191829 22188 consensus_queue.cc:260] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.191937 22188 ts_tablet_manager.cc:1428] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.193077 22187 tablet_bootstrap.cc:492] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: No bootstrap required, opened a new log
I20260619 20:44:53.193136 22187 ts_tablet_manager.cc:1397] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:44:53.193271 22187 raft_consensus.cc:348] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.193346 22187 raft_consensus.cc:374] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.193368 22187 raft_consensus.cc:729] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.193410 22187 consensus_queue.cc:260] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.193516 22187 ts_tablet_manager.cc:1428] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:53.198917 22191 tablet_bootstrap.cc:492] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: No bootstrap required, opened a new log
I20260619 20:44:53.198987 22191 ts_tablet_manager.cc:1397] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260619 20:44:53.199127 22191 raft_consensus.cc:348] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.199208 22191 raft_consensus.cc:374] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.199230 22191 raft_consensus.cc:729] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5e1d279f584e8280e2246d18032763, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.199275 22191 consensus_queue.cc:260] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.199389 22191 ts_tablet_manager.cc:1428] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 20:44:53.205435 22020 tablet.cc:2369] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:44:53.267037 22237 raft_consensus.cc:482] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:53.267123 22237 raft_consensus.cc:504] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.267302 22237 leader_election.cc:288] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857), 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595)
I20260619 20:44:53.267532 22113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8846a61da08b4e73bb8c540326c59216" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ff8c92ad534e0f85971b7571c037f3" is_pre_election: true
I20260619 20:44:53.267549 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8846a61da08b4e73bb8c540326c59216" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d" is_pre_election: true
I20260619 20:44:53.267645 22113 raft_consensus.cc:2454] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 0.
I20260619 20:44:53.267660 21973 raft_consensus.cc:2454] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 0.
I20260619 20:44:53.267820 21766 leader_election.cc:302] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:53.267908 22237 raft_consensus.cc:2781] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:53.267944 22237 raft_consensus.cc:482] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:53.267974 22237 raft_consensus.cc:3037] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.268935 22237 raft_consensus.cc:504] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.269079 22237 leader_election.cc:288] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 election: Requested vote from peers 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857), 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595)
I20260619 20:44:53.269240 22113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8846a61da08b4e73bb8c540326c59216" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ff8c92ad534e0f85971b7571c037f3"
I20260619 20:44:53.269335 22113 raft_consensus.cc:3037] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.269337 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8846a61da08b4e73bb8c540326c59216" candidate_uuid: "4f5e1d279f584e8280e2246d18032763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d"
I20260619 20:44:53.269415 21973 raft_consensus.cc:3037] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.270135 21973 raft_consensus.cc:2454] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 1.
I20260619 20:44:53.270135 22113 raft_consensus.cc:2454] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f5e1d279f584e8280e2246d18032763 in term 1.
I20260619 20:44:53.270323 21766 leader_election.cc:302] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:53.270443 22237 raft_consensus.cc:2781] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:53.270515 22237 raft_consensus.cc:686] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 1 LEADER]: Becoming Leader. State: Replica: 4f5e1d279f584e8280e2246d18032763, State: Running, Role: LEADER
I20260619 20:44:53.270568 22237 consensus_queue.cc:237] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.271181 21675 catalog_manager.cc:5495] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f5e1d279f584e8280e2246d18032763 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "4f5e1d279f584e8280e2246d18032763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } health_report { overall_health: UNKNOWN } } }
I20260619 20:44:53.288371 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.285657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.13.195.65:38443)
0619 20:44:53.285707 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.285709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.285719 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:53.285801 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.285802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.285816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.285908 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:53.286062 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.286695 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.287580 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.287789 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.287872 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.288055 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.288104 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.288174 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.288243 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.288265 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:44:53.289227 21973 raft_consensus.cc:1261] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Refusing update from remote peer 4f5e1d279f584e8280e2246d18032763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.289402 22113 raft_consensus.cc:1261] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Refusing update from remote peer 4f5e1d279f584e8280e2246d18032763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.289587 22237 consensus_queue.cc:1035] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.289708 22216 consensus_queue.cc:1035] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.292541 21676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:53.292623 21676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:53.293257 21676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Sending DeleteTablet for 3 replicas of tablet 8846a61da08b4e73bb8c540326c59216
I20260619 20:44:53.293587 21813 tablet_service.cc:1514] Processing DeleteTablet for tablet 8846a61da08b4e73bb8c540326c59216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:35740
I20260619 20:44:53.293587 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 8846a61da08b4e73bb8c540326c59216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:57666
I20260619 20:44:53.293702 22209 tablet_replica.cc:330] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: stopping tablet replica
I20260619 20:44:53.293696 22093 tablet_service.cc:1514] Processing DeleteTablet for tablet 8846a61da08b4e73bb8c540326c59216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:52202
I20260619 20:44:53.293773 22211 tablet_replica.cc:330] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: stopping tablet replica
I20260619 20:44:53.293769 22209 raft_consensus.cc:2229] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:53.293829 22209 raft_consensus.cc:2258] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.293824 22211 raft_consensus.cc:2229] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:53.293879 22211 raft_consensus.cc:2258] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.293907 22210 tablet_replica.cc:330] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: stopping tablet replica
I20260619 20:44:53.293963 22210 raft_consensus.cc:2229] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:53.294014 22210 raft_consensus.cc:2258] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.294304 22210 ts_tablet_manager.cc:1905] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.294283 21676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:44:53.294516 21676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:44:53.294931 22211 ts_tablet_manager.cc:1905] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.295678 21813 tablet_service.cc:1467] Processing CreateTablet for tablet 56e32279ca1e44b5bf1498eb279db6e2 (DEFAULT_TABLE table=test-table [id=90979c8860f74d80a0ed652eabde9582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:53.295722 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 56e32279ca1e44b5bf1498eb279db6e2 (DEFAULT_TABLE table=test-table [id=90979c8860f74d80a0ed652eabde9582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:53.295789 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e32279ca1e44b5bf1498eb279db6e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.295825 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e32279ca1e44b5bf1498eb279db6e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.295956 22209 ts_tablet_manager.cc:1905] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.296460 22211 ts_tablet_manager.cc:1918] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:53.296519 22211 log.cc:1196] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/8846a61da08b4e73bb8c540326c59216
I20260619 20:44:53.296738 22211 ts_tablet_manager.cc:1939] T 8846a61da08b4e73bb8c540326c59216 P 92ff8c92ad534e0f85971b7571c037f3: Deleting consensus metadata
I20260619 20:44:53.297194 22191 tablet_bootstrap.cc:492] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763: Bootstrap starting.
I20260619 20:44:53.297222 22188 tablet_bootstrap.cc:492] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: Bootstrap starting.
I20260619 20:44:53.295822 22210 ts_tablet_manager.cc:1918] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:53.297353 22210 log.cc:1196] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/8846a61da08b4e73bb8c540326c59216
I20260619 20:44:53.297358 22209 ts_tablet_manager.cc:1918] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:44:53.297402 22209 log.cc:1196] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/8846a61da08b4e73bb8c540326c59216
I20260619 20:44:53.297518 22191 tablet_bootstrap.cc:654] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.297565 22188 tablet_bootstrap.cc:654] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.297588 22210 ts_tablet_manager.cc:1939] T 8846a61da08b4e73bb8c540326c59216 P 5122bb8404ad4ac5b33d80b043a6316d: Deleting consensus metadata
I20260619 20:44:53.297591 22209 ts_tablet_manager.cc:1939] T 8846a61da08b4e73bb8c540326c59216 P 4f5e1d279f584e8280e2246d18032763: Deleting consensus metadata
I20260619 20:44:53.297907 21660 catalog_manager.cc:4841] TS 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857): tablet 8846a61da08b4e73bb8c540326c59216 (table test-table [id=041fc72dc3bc4f388e633aaf15485894]) successfully deleted
I20260619 20:44:53.298066 21662 catalog_manager.cc:4841] TS 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595): tablet 8846a61da08b4e73bb8c540326c59216 (table test-table [id=041fc72dc3bc4f388e633aaf15485894]) successfully deleted
I20260619 20:44:53.298084 22093 tablet_service.cc:1467] Processing CreateTablet for tablet 56e32279ca1e44b5bf1498eb279db6e2 (DEFAULT_TABLE table=test-table [id=90979c8860f74d80a0ed652eabde9582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:44:53.298158 21662 catalog_manager.cc:4841] TS 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443): tablet 8846a61da08b4e73bb8c540326c59216 (table test-table [id=041fc72dc3bc4f388e633aaf15485894]) successfully deleted
I20260619 20:44:53.298173 22093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e32279ca1e44b5bf1498eb279db6e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:53.298213 22191 tablet_bootstrap.cc:492] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763: No bootstrap required, opened a new log
I20260619 20:44:53.298271 22191 ts_tablet_manager.cc:1397] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 20:44:53.298370 22191 raft_consensus.cc:348] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.298456 22191 raft_consensus.cc:374] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.298483 22191 raft_consensus.cc:729] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5e1d279f584e8280e2246d18032763, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.298545 22191 consensus_queue.cc:260] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.298591 22188 tablet_bootstrap.cc:492] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: No bootstrap required, opened a new log
I20260619 20:44:53.298628 22188 ts_tablet_manager.cc:1397] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:53.298646 22191 ts_tablet_manager.cc:1428] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.298753 22188 raft_consensus.cc:348] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.298820 22188 raft_consensus.cc:374] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.298839 22188 raft_consensus.cc:729] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5122bb8404ad4ac5b33d80b043a6316d, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.298902 22188 consensus_queue.cc:260] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.298993 22188 ts_tablet_manager.cc:1428] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.299930 22187 tablet_bootstrap.cc:492] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: Bootstrap starting.
I20260619 20:44:53.300305 22187 tablet_bootstrap.cc:654] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:53.300879 22187 tablet_bootstrap.cc:492] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: No bootstrap required, opened a new log
I20260619 20:44:53.300918 22187 ts_tablet_manager.cc:1397] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 20:44:53.300990 22187 raft_consensus.cc:348] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.301048 22187 raft_consensus.cc:374] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:53.301065 22187 raft_consensus.cc:729] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Initialized, Role: FOLLOWER
I20260619 20:44:53.301132 22187 consensus_queue.cc:260] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.301231 22187 ts_tablet_manager.cc:1428] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:53.425427 22193 raft_consensus.cc:482] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:44:53.425562 22193 raft_consensus.cc:504] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.425762 22193 leader_election.cc:288] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443), 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857)
I20260619 20:44:53.426002 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e32279ca1e44b5bf1498eb279db6e2" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d" is_pre_election: true
I20260619 20:44:53.426002 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e32279ca1e44b5bf1498eb279db6e2" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5e1d279f584e8280e2246d18032763" is_pre_election: true
I20260619 20:44:53.426115 21833 raft_consensus.cc:2454] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 0.
I20260619 20:44:53.426115 21973 raft_consensus.cc:2454] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 0.
I20260619 20:44:53.426276 22044 leader_election.cc:302] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5122bb8404ad4ac5b33d80b043a6316d, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:53.426383 22193 raft_consensus.cc:2781] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:44:53.426445 22193 raft_consensus.cc:482] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:44:53.426469 22193 raft_consensus.cc:3037] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.427096 22193 raft_consensus.cc:504] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.427235 22193 leader_election.cc:288] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443), 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857)
I20260619 20:44:53.427388 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e32279ca1e44b5bf1498eb279db6e2" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5e1d279f584e8280e2246d18032763"
I20260619 20:44:53.427403 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e32279ca1e44b5bf1498eb279db6e2" candidate_uuid: "92ff8c92ad534e0f85971b7571c037f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5122bb8404ad4ac5b33d80b043a6316d"
I20260619 20:44:53.427464 21973 raft_consensus.cc:3037] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.427464 21833 raft_consensus.cc:3037] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:53.428176 21973 raft_consensus.cc:2454] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 1.
I20260619 20:44:53.428177 21833 raft_consensus.cc:2454] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ff8c92ad534e0f85971b7571c037f3 in term 1.
I20260619 20:44:53.428345 22046 leader_election.cc:302] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5e1d279f584e8280e2246d18032763, 92ff8c92ad534e0f85971b7571c037f3; no voters:
I20260619 20:44:53.428476 22193 raft_consensus.cc:2781] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:53.428545 22193 raft_consensus.cc:686] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 LEADER]: Becoming Leader. State: Replica: 92ff8c92ad534e0f85971b7571c037f3, State: Running, Role: LEADER
I20260619 20:44:53.428604 22193 consensus_queue.cc:237] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } }
I20260619 20:44:53.429153 21675 catalog_manager.cc:5495] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "92ff8c92ad534e0f85971b7571c037f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ff8c92ad534e0f85971b7571c037f3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40595 } health_report { overall_health: HEALTHY } } }
I20260619 20:44:53.448441 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.446517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.13.195.67:40595)
0619 20:44:53.446561 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.446563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.446601 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:44:53.446666 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.446666 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.446675 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.446719 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:53.446799 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.447261 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.447933 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.448122 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.448172 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.448280 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.448304 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.448335 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.448368 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.448382 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:44:53.452265 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.450338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:42559 (local address 127.13.195.67:40595)
0619 20:44:53.450370 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:53.450371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:53.450418 (+ 47us) server_negotiation.cc:407] Connection header received
0619 20:44:53.450519 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:53.450520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:53.450527 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:53.450573 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:44:53.450686 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.451103 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.451742 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.451907 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.451957 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.452084 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.452105 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.452143 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:53.452177 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 20:44:53.452190 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 20:44:53.452278 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.450285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40595 (local address 127.13.195.67:42559)
0619 20:44:53.450374 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.450384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:53.450466 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:53.450582 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:53.450583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:53.450584 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:44:53.450649 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.450650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.451161 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.451161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.451675 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:53.451676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:53.451970 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:53.451971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:53.452005 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:53.452009 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:44:53.452144 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:44:53.452145 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:44:53.452175 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 20:44:53.452204 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":31}
I20260619 20:44:53.454064 21973 raft_consensus.cc:1261] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Refusing update from remote peer 92ff8c92ad534e0f85971b7571c037f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.454083 21833 raft_consensus.cc:1261] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Refusing update from remote peer 92ff8c92ad534e0f85971b7571c037f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:44:53.454274 22238 consensus_queue.cc:1035] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5122bb8404ad4ac5b33d80b043a6316d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 41857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.454376 22193 consensus_queue.cc:1035] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f5e1d279f584e8280e2246d18032763" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:44:53.463791 21675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:53.463876 21675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:44:53.464452 21675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 880c8ac1a6e84254bbb182cdf5b557da: Sending DeleteTablet for 3 replicas of tablet 56e32279ca1e44b5bf1498eb279db6e2
I20260619 20:44:53.464695 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e32279ca1e44b5bf1498eb279db6e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:57666
I20260619 20:44:53.464720 21813 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e32279ca1e44b5bf1498eb279db6e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:35740
I20260619 20:44:53.464788 22210 tablet_replica.cc:330] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: stopping tablet replica
I20260619 20:44:53.464823 22209 tablet_replica.cc:330] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763: stopping tablet replica
I20260619 20:44:53.464838 22210 raft_consensus.cc:2229] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:53.464867 22210 raft_consensus.cc:2258] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.464874 22209 raft_consensus.cc:2229] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:44:53.464920 22209 raft_consensus.cc:2258] T 56e32279ca1e44b5bf1498eb279db6e2 P 4f5e1d279f584e8280e2246d18032763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.465068 22093 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e32279ca1e44b5bf1498eb279db6e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:44:53 UTC) from {username='oryx', principal='oryx/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:52202
I20260619 20:44:53.465152 22211 tablet_replica.cc:330] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: stopping tablet replica
I20260619 20:44:53.465201 22211 raft_consensus.cc:2229] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:44:53.465225 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 21748
I20260619 20:44:53.465257 22211 raft_consensus.cc:2258] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:44:53.467231 22210 ts_tablet_manager.cc:1905] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.468780 22210 ts_tablet_manager.cc:1918] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:44:53.468837 22210 log.cc:1196] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/56e32279ca1e44b5bf1498eb279db6e2
I20260619 20:44:53.469069 22210 ts_tablet_manager.cc:1939] T 56e32279ca1e44b5bf1498eb279db6e2 P 5122bb8404ad4ac5b33d80b043a6316d: Deleting consensus metadata
I20260619 20:44:53.469389 21660 catalog_manager.cc:4841] TS 5122bb8404ad4ac5b33d80b043a6316d (127.13.195.66:41857): tablet 56e32279ca1e44b5bf1498eb279db6e2 (table test-table [id=90979c8860f74d80a0ed652eabde9582]) successfully deleted
I20260619 20:44:53.470667 22211 ts_tablet_manager.cc:1905] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:44:53.471944 22211 ts_tablet_manager.cc:1918] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
W20260619 20:44:53.471968 21662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:38443 (error 108)
I20260619 20:44:53.472007 22211 log.cc:1196] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/56e32279ca1e44b5bf1498eb279db6e2
I20260619 20:44:53.472178 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 21888
I20260619 20:44:53.472266 22211 ts_tablet_manager.cc:1939] T 56e32279ca1e44b5bf1498eb279db6e2 P 92ff8c92ad534e0f85971b7571c037f3: Deleting consensus metadata
I20260619 20:44:53.472961 21662 catalog_manager.cc:4841] TS 92ff8c92ad534e0f85971b7571c037f3 (127.13.195.67:40595): tablet 56e32279ca1e44b5bf1498eb279db6e2 (table test-table [id=90979c8860f74d80a0ed652eabde9582]) successfully deleted
I20260619 20:44:53.473104 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:53.472891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38443 (local address 127.0.0.1:35768)
0619 20:44:53.472994 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:53.473052 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":41,"threads_started":1}
W20260619 20:44:53.473277 21662 catalog_manager.cc:4584] TS 4f5e1d279f584e8280e2246d18032763 (127.13.195.65:38443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56e32279ca1e44b5bf1498eb279db6e2: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38443: connect: Connection refused (error 111)
I20260619 20:44:53.478157 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 22028
I20260619 20:44:53.484053 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 21644
2026-06-19T20:44:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2540 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[22272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[22272](info): set up 2 sockets
Jun 19 20:44:53 dist-test-slave-n2td krb5kdc[22272](info): commencing operation
krb5kdc: starting...
W20260619 20:44:54.526036 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:54 dist-test-slave-n2td krb5kdc[22272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:54.645010 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44031
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:44031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:54.725324 22285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:54.725528 22285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:54.725548 22285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:54.726675 22285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:54.726709 22285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:54.726724 22285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:54.726737 22285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:54.726748 22285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:54.728150 22285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:44031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:54.728348 22285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:54.728523 22285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:54.733295 22290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:54.733302 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:54.733315 22293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:54.733464 22285 server_base.cc:1034] running on GCE node
I20260619 20:44:54.733915 22285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:54.734162 22285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:54.735297 22285 hybrid_clock.cc:648] HybridClock initialized: now 1781901894735264 us; error 57 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T20:44:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1257 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:44:54 dist-test-slave-n2td krb5kdc[22298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:44:54 dist-test-slave-n2td krb5kdc[22298](info): set up 2 sockets
Jun 19 20:44:54 dist-test-slave-n2td krb5kdc[22298](info): commencing operation
krb5kdc: starting...
W20260619 20:44:55.789897 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:44:55 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:44:55.909518 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36881
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:36881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:55.991554 22311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:55.991763 22311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:55.991786 22311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:55.992960 22311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:55.993014 22311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:55.993029 22311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:55.993085 22311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:55.993104 22311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:55.994697 22311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:36881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:55.994948 22311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:55.995188 22311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:55.999953 22319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:55.999970 22316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:55.999923 22317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:56.000177 22311 server_base.cc:1034] running on GCE node
I20260619 20:44:56.000448 22311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:56.000764 22311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:56.001909 22311 hybrid_clock.cc:648] HybridClock initialized: now 1781901896001887 us; error 37 us; skew 500 ppm
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:56.011258 22311 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:56.011700 22311 webserver.cc:460] Webserver started at http://127.13.195.126:42477/ using document root <none> and password file <none>
I20260619 20:44:56.011909 22311 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:56.011950 22311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:56.012053 22311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:56.013079 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "6feb9e50d7d74343a77c09fb7ef69279"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "5449536ed22c12ebd5d63f32d79c12e6"
server_key_iv: "95db49f739311a6d49b6a87e48175a73"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.013372 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "6feb9e50d7d74343a77c09fb7ef69279"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "5449536ed22c12ebd5d63f32d79c12e6"
server_key_iv: "95db49f739311a6d49b6a87e48175a73"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.014663 22311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:44:56.015339 22326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.015722 22311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:44:56.015820 22311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "6feb9e50d7d74343a77c09fb7ef69279"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "5449536ed22c12ebd5d63f32d79c12e6"
server_key_iv: "95db49f739311a6d49b6a87e48175a73"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.015884 22311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:56.039324 22311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:56.041326 22311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:56.041484 22311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:56.045558 22311 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:36881
I20260619 20:44:56.045619 22378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:36881 every 8 connection(s)
I20260619 20:44:56.045964 22311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:56.046883 22379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:44:56.049084 22379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Bootstrap starting.
I20260619 20:44:56.049610 22379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Neither blocks nor log segments found. Creating new log.
I20260619 20:44:56.049858 22379 log.cc:824] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:56.050603 22379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: No bootstrap required, opened a new log
I20260619 20:44:56.052102 22379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.052240 22379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:44:56.052263 22379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6feb9e50d7d74343a77c09fb7ef69279, State: Initialized, Role: FOLLOWER
I20260619 20:44:56.052400 22379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.052479 22379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:56.052501 22379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:56.052536 22379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:44:56.053005 22379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.053084 22379 leader_election.cc:302] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6feb9e50d7d74343a77c09fb7ef69279; no voters:
I20260619 20:44:56.053210 22379 leader_election.cc:288] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:44:56.053277 22382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:44:56.053449 22382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 LEADER]: Becoming Leader. State: Replica: 6feb9e50d7d74343a77c09fb7ef69279, State: Running, Role: LEADER
I20260619 20:44:56.053470 22379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:56.053536 22382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.053934 22384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6feb9e50d7d74343a77c09fb7ef69279. Latest consensus state: current_term: 1 leader_uuid: "6feb9e50d7d74343a77c09fb7ef69279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } } }
I20260619 20:44:56.054061 22384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:56.053910 22383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6feb9e50d7d74343a77c09fb7ef69279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } } }
I20260619 20:44:56.054167 22383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:56.054507 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 22311
I20260619 20:44:56.054502 22386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:56.054617 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:44:56.054824 14093 external_mini_cluster.cc:1428] Setting key 7e637944f80638c1fffc1518fdb638cc
I20260619 20:44:56.055055 22386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:56.056692 22386 catalog_manager.cc:1347] Generated new cluster ID: 59b498a875ee44c09c216d129be2ff06
I20260619 20:44:56.056741 22386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:56.072444 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.055468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.13.195.126:36881)
0619 20:44:56.055694 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:56.055703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:56.055785 (+ 82us) server_negotiation.cc:407] Connection header received
0619 20:44:56.055926 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:56.055932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:56.055973 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:56.056122 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:56.062811 (+ 6689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.064287 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.065319 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.065986 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.066101 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.068880 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:56.068884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:56.068884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:56.068897 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:56.070827 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.071227 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.071230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.071231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.071333 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.071532 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.071532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.071533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.071821 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:56.071963 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:56.072081 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 20:44:56.072208 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260619 20:44:56.074860 22386 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:44:56.075354 22386 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:56.080063 22386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Generated new TSK 0
I20260619 20:44:56.080248 22386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:44:56.113296 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36881
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:56.194312 22405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:56.194588 22405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:56.194630 22405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:56.195837 22405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:56.195907 22405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:56.195953 22405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:56.197535 22405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:56.197808 22405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:56.198027 22405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:56.202987 22411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.203006 22410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.203006 22413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:56.203246 22405 server_base.cc:1034] running on GCE node
I20260619 20:44:56.203454 22405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:56.203709 22405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:56.204864 22405 hybrid_clock.cc:648] HybridClock initialized: now 1781901896204854 us; error 39 us; skew 500 ppm
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:56.207362 22405 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:44:56.207835 22405 webserver.cc:460] Webserver started at http://127.13.195.65:40323/ using document root <none> and password file <none>
I20260619 20:44:56.208055 22405 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:56.208127 22405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:56.208271 22405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:56.209407 22405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "72ffd86573ab444a847d9710eee6c5e9"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "6a79e6611a054623ff887e8d277f147c"
server_key_iv: "cb2607e93c9b45a760568393307f775a"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.209779 22405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "72ffd86573ab444a847d9710eee6c5e9"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "6a79e6611a054623ff887e8d277f147c"
server_key_iv: "cb2607e93c9b45a760568393307f775a"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.211179 22405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:44:56.211907 22420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.212067 22405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:56.212165 22405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "72ffd86573ab444a847d9710eee6c5e9"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "6a79e6611a054623ff887e8d277f147c"
server_key_iv: "cb2607e93c9b45a760568393307f775a"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.212237 22405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:56.237385 22405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:56.239287 22405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:56.239449 22405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:56.239711 22405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:56.240011 22405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:56.240062 22405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.240103 22405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:56.240142 22405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.245821 22405 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:45121
I20260619 20:44:56.245862 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:45121 every 8 connection(s)
I20260619 20:44:56.246168 22405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:56.248732 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 22405
I20260619 20:44:56.248852 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:44:56.249037 14093 external_mini_cluster.cc:1428] Setting key 4053cc4b302f6c09d5a254a70d553e56
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:56.256618 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.246977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:42577 (local address 127.13.195.126:36881)
0619 20:44:56.247033 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:56.247034 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:56.247227 (+ 193us) server_negotiation.cc:407] Connection header received
0619 20:44:56.247698 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:56.247699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:56.247711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:56.247769 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:56.248722 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.249426 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.250903 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.251100 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.251179 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.253316 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:56.253318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:56.253318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:56.253327 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:56.255168 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.255628 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.255630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.255630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.255694 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.255929 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.255930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.255930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.256090 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:56.256157 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:56.256403 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 20:44:56.256501 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:44:56.256650 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.246944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:42577)
0619 20:44:56.247104 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:56.247145 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:56.247619 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:56.247820 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:56.247823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:56.248386 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:56.248633 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.248640 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.249503 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.249504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.250809 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.250812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.251186 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.251187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.251311 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.251803 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:56.251811 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:56.253209 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:56.255245 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:56.255252 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:56.255254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:56.255547 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:56.255761 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:56.255761 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:56.255761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:56.255864 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:56.256160 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:56.256164 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:56.256280 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 20:44:56.256358 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:44:56.256476 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260619 20:44:56.257194 22534 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36881
I20260619 20:44:56.257298 22534 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:56.257484 22534 heartbeater.cc:511] Master 127.13.195.126:36881 requested a full tablet report, sending...
I20260619 20:44:56.258036 22343 ts_manager.cc:194] Registered new tserver with Master: 72ffd86573ab444a847d9710eee6c5e9 (127.13.195.65:45121)
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
I20260619 20:44:56.258757 22343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:42577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:44:56.274717 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36881
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:56.355335 22539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:56.355540 22539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:56.355562 22539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:56.356685 22539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:56.356725 22539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:56.356774 22539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:56.358152 22539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:56.358394 22539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:56.358630 22539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:56.363373 22547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.363418 22544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.363374 22545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:56.363677 22539 server_base.cc:1034] running on GCE node
I20260619 20:44:56.363850 22539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:56.364041 22539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:56.365180 22539 hybrid_clock.cc:648] HybridClock initialized: now 1781901896365159 us; error 38 us; skew 500 ppm
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:56.367628 22539 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:44:56.368094 22539 webserver.cc:460] Webserver started at http://127.13.195.66:35093/ using document root <none> and password file <none>
I20260619 20:44:56.368321 22539 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:56.368366 22539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:56.368530 22539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:56.369565 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "af2fab2690da493793ac91a7c95b24a1"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "51a41804c513675ee8bea8ccf2243f53"
server_key_iv: "bce3010ad5db845923bda4ae5c1d17fb"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.369920 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "af2fab2690da493793ac91a7c95b24a1"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "51a41804c513675ee8bea8ccf2243f53"
server_key_iv: "bce3010ad5db845923bda4ae5c1d17fb"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.371256 22539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:44:56.371941 22554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.372118 22539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:56.372207 22539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "af2fab2690da493793ac91a7c95b24a1"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "51a41804c513675ee8bea8ccf2243f53"
server_key_iv: "bce3010ad5db845923bda4ae5c1d17fb"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.372260 22539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:56.385667 22539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:56.387991 22539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:56.388126 22539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:56.388309 22539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:56.388612 22539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:56.388648 22539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.388689 22539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:56.388715 22539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.394229 22539 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:33093
I20260619 20:44:56.394285 22667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:33093 every 8 connection(s)
I20260619 20:44:56.394609 22539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:56.399526 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 22539
I20260619 20:44:56.399667 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:44:56.399856 14093 external_mini_cluster.cc:1428] Setting key 7b8e322eef394d74c29482e6d80e1579
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:56.404254 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.395297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:58769 (local address 127.13.195.126:36881)
0619 20:44:56.395357 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:56.395359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:56.395555 (+ 196us) server_negotiation.cc:407] Connection header received
0619 20:44:56.396087 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:56.396089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:56.396103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:56.396153 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:56.397157 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.397651 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.398756 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.398951 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.399040 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.401403 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:56.401404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:56.401405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:56.401412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:56.402867 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.403289 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.403290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.403291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.403348 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.403609 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.403610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.403610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.403726 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:56.403794 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:56.404082 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 20:44:56.404164 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:44:56.404436 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.395266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:58769)
0619 20:44:56.395412 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:56.395454 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:56.395979 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:56.396223 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:56.396227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:56.396854 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:56.397049 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.397054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.397740 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.397741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.398655 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.398657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.399050 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.399050 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.399180 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.399797 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:56.399805 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:56.401288 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:56.402965 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:56.402972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:56.402975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:56.403200 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:56.403412 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:56.403412 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:56.403413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:56.403534 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:56.403797 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:56.403804 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:56.403943 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 20:44:56.404041 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 20:44:56.404200 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260619 20:44:56.406518 22668 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36881
I20260619 20:44:56.406615 22668 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:56.406811 22668 heartbeater.cc:511] Master 127.13.195.126:36881 requested a full tablet report, sending...
I20260619 20:44:56.407223 22343 ts_manager.cc:194] Registered new tserver with Master: af2fab2690da493793ac91a7c95b24a1 (127.13.195.66:33093)
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:44:56.407636 22343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:58769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:44:56.424937 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36881
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:56.506388 22673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:56.506639 22673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:56.506665 22673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:56.507803 22673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:56.507841 22673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:56.507869 22673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:56.509338 22673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:56.509538 22673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:56.509723 22673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:56.514804 22678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.514844 22681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.514816 22679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:56.514946 22673 server_base.cc:1034] running on GCE node
I20260619 20:44:56.515266 22673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:56.515554 22673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:56.516716 22673 hybrid_clock.cc:648] HybridClock initialized: now 1781901896516700 us; error 44 us; skew 500 ppm
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:56.519191 22673 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:44:56.519675 22673 webserver.cc:460] Webserver started at http://127.13.195.67:46747/ using document root <none> and password file <none>
I20260619 20:44:56.519894 22673 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:56.519956 22673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:56.520093 22673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:44:56.521179 22673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "1d84a612edb64b8d812b6baa43a17352"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "474304f458a8f3e95ae70efed3784f38"
server_key_iv: "eb7c7a9d0586a81442fa49f428e149ee"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.521540 22673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "1d84a612edb64b8d812b6baa43a17352"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "474304f458a8f3e95ae70efed3784f38"
server_key_iv: "eb7c7a9d0586a81442fa49f428e149ee"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.522866 22673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:44:56.523600 22688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.523753 22673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:44:56.523850 22673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "1d84a612edb64b8d812b6baa43a17352"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "474304f458a8f3e95ae70efed3784f38"
server_key_iv: "eb7c7a9d0586a81442fa49f428e149ee"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.523917 22673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:56.548539 22673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:56.550390 22673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:56.550568 22673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:56.550815 22673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:56.551105 22673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:56.551154 22673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.551194 22673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:56.551222 22673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.556968 22673 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:37649
I20260619 20:44:56.557049 22801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:37649 every 8 connection(s)
I20260619 20:44:56.557345 22673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:56.559826 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 22673
I20260619 20:44:56.559958 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:44:56.560140 14093 external_mini_cluster.cc:1428] Setting key 6d692ede7282d9c370cd24d4f9526512
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:44:56.565996 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.558054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:48657 (local address 127.13.195.126:36881)
0619 20:44:56.558098 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:56.558100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:56.558304 (+ 204us) server_negotiation.cc:407] Connection header received
0619 20:44:56.558833 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:56.558834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:56.558844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:56.558885 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:56.559873 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.560505 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.561628 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.561809 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.561862 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.563814 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:56.563815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:56.563815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:56.563821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:56.564933 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.565277 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.565278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.565278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.565323 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:56.565516 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:56.565517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:56.565517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:56.565610 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:56.565657 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:56.565870 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:44:56.565926 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:44:56.566191 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.558034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:48657)
0619 20:44:56.558189 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:56.558223 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:56.558737 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:56.558938 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:56.558941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:56.559511 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:56.559756 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.559763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.560583 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.560584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.561557 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.561560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.561874 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.561875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.561982 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.562409 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:56.562417 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:56.563728 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:44:56.564998 (+ 1270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:56.565003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:56.565005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:56.565218 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:56.565376 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:44:56.565376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:44:56.565376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:44:56.565468 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:44:56.565658 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:44:56.565661 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:44:56.565766 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 20:44:56.565847 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:44:56.565957 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260619 20:44:56.566654 22802 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36881
I20260619 20:44:56.566751 22802 heartbeater.cc:464] Registering TS with master...
I20260619 20:44:56.566917 22802 heartbeater.cc:511] Master 127.13.195.126:36881 requested a full tablet report, sending...
I20260619 20:44:56.567266 22343 ts_manager.cc:194] Registered new tserver with Master: 1d84a612edb64b8d812b6baa43a17352 (127.13.195.67:37649)
I20260619 20:44:56.567625 22343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:48657
I20260619 20:44:56.571770 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab.
Entry for principal oryx/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab.
WARNING: no policy specified for oryx/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab.
Entry for principal oryx/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab.
WARNING: no policy specified for oryx/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab.
Entry for principal oryx/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab.
I20260619 20:44:56.641708 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 22405
I20260619 20:44:56.646265 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 22539
I20260619 20:44:56.650777 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 22673
I20260619 20:44:56.655048 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 22311
I20260619 20:44:56.659878 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36881
--webserver_interface=127.13.195.126
--webserver_port=42477
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:36881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:44:56.739594 22811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:56.739852 22811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:56.739890 22811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:56.741050 22811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:44:56.741122 22811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:56.741147 22811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:56.741174 22811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:44:56.741204 22811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:44:56.742699 22811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:36881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=42477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:56.742983 22811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:56.743207 22811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:56.747927 22817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.747895 22819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.747911 22816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:56.748114 22811 server_base.cc:1034] running on GCE node
I20260619 20:44:56.748327 22811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:56.748565 22811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:56.749697 22811 hybrid_clock.cc:648] HybridClock initialized: now 1781901896749688 us; error 26 us; skew 500 ppm
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:56.752068 22811 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:44:56.752548 22811 webserver.cc:460] Webserver started at http://127.13.195.126:42477/ using document root <none> and password file <none>
I20260619 20:44:56.752779 22811 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:56.752849 22811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:56.754145 22811 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:56.754837 22826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.755009 22811 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.755102 22811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "6feb9e50d7d74343a77c09fb7ef69279"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "5449536ed22c12ebd5d63f32d79c12e6"
server_key_iv: "95db49f739311a6d49b6a87e48175a73"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.755445 22811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:56.784147 22811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:56.786231 22811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:56.786361 22811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:56.790251 22811 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:36881
I20260619 20:44:56.790320 22878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:36881 every 8 connection(s)
I20260619 20:44:56.790719 22811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:44:56.792182 22879 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:44:56.792870 22879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Bootstrap starting.
I20260619 20:44:56.794740 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 22811
I20260619 20:44:56.795023 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:45121
--local_ip_for_outbound_sockets=127.13.195.65
--tserver_master_addrs=127.13.195.126:36881
--webserver_port=40323
--webserver_interface=127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:44:56.801815 22879 log.cc:824] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Log is configured to *not* fsync() on all Append() calls
I20260619 20:44:56.803591 22879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:44:56.803879 22879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Bootstrap complete.
I20260619 20:44:56.806139 22879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.806389 22879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6feb9e50d7d74343a77c09fb7ef69279, State: Initialized, Role: FOLLOWER
I20260619 20:44:56.806602 22879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.806695 22879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:44:56.806751 22879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:44:56.806798 22879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:44:56.807797 22879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.807915 22879 leader_election.cc:302] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6feb9e50d7d74343a77c09fb7ef69279; no voters:
I20260619 20:44:56.808120 22879 leader_election.cc:288] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:44:56.808156 22883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:44:56.808270 22883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [term 2 LEADER]: Becoming Leader. State: Replica: 6feb9e50d7d74343a77c09fb7ef69279, State: Running, Role: LEADER
I20260619 20:44:56.808359 22883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } }
I20260619 20:44:56.808403 22879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:44:56.808602 22883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6feb9e50d7d74343a77c09fb7ef69279. Latest consensus state: current_term: 2 leader_uuid: "6feb9e50d7d74343a77c09fb7ef69279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } } }
I20260619 20:44:56.808701 22883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:56.808602 22884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6feb9e50d7d74343a77c09fb7ef69279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6feb9e50d7d74343a77c09fb7ef69279" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36881 } } }
I20260619 20:44:56.808897 22884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279 [sys.catalog]: This master's current role is: LEADER
I20260619 20:44:56.809239 22887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:44:56.809759 22887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:44:56.809890 22887 catalog_manager.cc:1259] Loaded cluster ID: 59b498a875ee44c09c216d129be2ff06
I20260619 20:44:56.809916 22887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:44:56.811331 22887 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:44:56.811510 22887 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6feb9e50d7d74343a77c09fb7ef69279: Loaded TSK: 0
I20260619 20:44:56.811724 22887 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:44:56.878742 22881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:56.878970 22881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:56.879009 22881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:56.880249 22881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:56.880311 22881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:56.880365 22881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:44:56.881800 22881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.65.keytab
--principal=oryx/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:45121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=40323
--tserver_master_addrs=127.13.195.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:56.882057 22881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:56.882287 22881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:56.887193 22908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.887296 22905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:56.887207 22906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:56.888063 22881 server_base.cc:1034] running on GCE node
I20260619 20:44:56.888391 22881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:56.888648 22881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:56.889809 22881 hybrid_clock.cc:648] HybridClock initialized: now 1781901896889797 us; error 34 us; skew 500 ppm
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901896, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:56.892711 22881 init.cc:377] Logged in from keytab as oryx/127.13.195.65@KRBTEST.COM (short username oryx)
I20260619 20:44:56.893311 22881 webserver.cc:460] Webserver started at http://127.13.195.65:40323/ using document root <none> and password file <none>
I20260619 20:44:56.893625 22881 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:56.893724 22881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:56.895691 22881 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:56.896436 22915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.896641 22881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:56.896751 22881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "72ffd86573ab444a847d9710eee6c5e9"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "6a79e6611a054623ff887e8d277f147c"
server_key_iv: "cb2607e93c9b45a760568393307f775a"
server_key_version: "encryptionkey@0"
I20260619 20:44:56.897241 22881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:56.922799 22881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:56.925457 22881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:56.925643 22881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:56.925894 22881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:56.926270 22881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:56.926322 22881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.926365 22881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:56.926393 22881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:56.933717 22881 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:45121
I20260619 20:44:56.933750 23028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:45121 every 8 connection(s)
I20260619 20:44:56.934172 22881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:44:56.940445 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 22881
I20260619 20:44:56.941007 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:33093
--local_ip_for_outbound_sockets=127.13.195.66
--tserver_master_addrs=127.13.195.126:36881
--webserver_port=35093
--webserver_interface=127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:56.943778 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.934913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:33161)
0619 20:44:56.935124 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:56.935170 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:56.935797 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:56.936216 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:56.936220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:56.936973 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:56.937261 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.937268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.938345 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.938346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.939631 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.939633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.940176 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.940177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.940321 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.940855 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:56.940865 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:56.943564 (+ 2699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
W20260619 20:44:56.943842 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:56.935038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:33161 (local address 127.13.195.126:36881)
0619 20:44:56.935218 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:56.935227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:56.935265 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:44:56.935983 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:56.935988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:56.936028 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:56.936134 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:56.937368 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.938246 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.939712 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.940087 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.940164 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.943685 (+ 3521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:33161: BlockingRecv error: recv got EOF from 127.13.195.65:33161 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
W20260619 20:44:56.944013 23029 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:36881 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.195.126:36881: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:56.948532 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.944266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:33131)
0619 20:44:56.944347 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:56.944359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:56.944401 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:56.944589 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:56.944590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:56.944717 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:56.944759 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.944760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.945449 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.945450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.945948 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.945950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.946327 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.946328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.946391 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.946853 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:56.946857 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:56.948446 (+ 1589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 20:44:56.948578 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:56.944312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:33131 (local address 127.13.195.126:36881)
0619 20:44:56.944475 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:56.944476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:56.944483 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:44:56.944502 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:56.944503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:56.944512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:56.944574 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:56.944835 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.945367 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.946048 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.946248 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.946327 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.948515 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:33131: BlockingRecv error: recv got EOF from 127.13.195.65:33131 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:56 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:56.952929 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:56.949027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:38883)
0619 20:44:56.949109 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:56.949120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:56.949165 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:56.949288 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:56.949289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:56.949411 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:56.949435 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.949436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.950034 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.950035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.950587 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:56.950589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.950915 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.950916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:56.950966 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.951357 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:56.951362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:56.952853 (+ 1491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 20:44:56.952977 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:56.949071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:38883 (local address 127.13.195.126:36881)
0619 20:44:56.949115 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:56.949117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:56.949152 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:44:56.949221 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:56.949222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:56.949232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:56.949284 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:56.949509 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.949947 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.950669 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:56.950853 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:56.950918 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:56.952913 (+ 1995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:38883: BlockingRecv error: recv got EOF from 127.13.195.65:38883 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 20:44:56.953303 23029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:44:57.029090 23033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:57.029292 23033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:57.029315 23033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:57.030419 23033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:57.030473 23033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:57.030499 23033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:44:57.031819 23033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.66.keytab
--principal=oryx/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:33093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=35093
--tserver_master_addrs=127.13.195.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:57.031994 23033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:57.032164 23033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:57.037330 23039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:57.037279 23041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:57.037518 23033 server_base.cc:1034] running on GCE node
W20260619 20:44:57.037281 23038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:57.037762 23033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:57.038020 23033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:57.039203 23033 hybrid_clock.cc:648] HybridClock initialized: now 1781901897039203 us; error 41 us; skew 500 ppm
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901897, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:57.041653 23033 init.cc:377] Logged in from keytab as oryx/127.13.195.66@KRBTEST.COM (short username oryx)
I20260619 20:44:57.042065 23033 webserver.cc:460] Webserver started at http://127.13.195.66:35093/ using document root <none> and password file <none>
I20260619 20:44:57.042305 23033 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:57.042382 23033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:57.043682 23033 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:44:57.044250 23048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:57.044409 23033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:44:57.044493 23033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "af2fab2690da493793ac91a7c95b24a1"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "51a41804c513675ee8bea8ccf2243f53"
server_key_iv: "bce3010ad5db845923bda4ae5c1d17fb"
server_key_version: "encryptionkey@0"
I20260619 20:44:57.044826 23033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:57.054179 23033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:57.055979 23033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:57.056133 23033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:57.056348 23033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:57.056634 23033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:57.056684 23033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:57.056725 23033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:57.056754 23033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:57.062567 23033 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:33093
I20260619 20:44:57.062637 23161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:33093 every 8 connection(s)
I20260619 20:44:57.063002 23033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:44:57.066879 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23033
I20260619 20:44:57.067183 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:37649
--local_ip_for_outbound_sockets=127.13.195.67
--tserver_master_addrs=127.13.195.126:36881
--webserver_port=46747
--webserver_interface=127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:57.070617 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.063675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:51395)
0619 20:44:57.063823 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:57.063868 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:57.064351 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:57.064612 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:57.064615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:57.065170 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:57.065353 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.065357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.066053 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.066054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.066933 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.066936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.067366 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.067367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.067498 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.068035 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:57.068046 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:57.070414 (+ 2368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
W20260619 20:44:57.070623 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:57.063673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:51395 (local address 127.13.195.126:36881)
0619 20:44:57.063732 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.063734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.063949 (+ 215us) server_negotiation.cc:407] Connection header received
0619 20:44:57.064452 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.064453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.064468 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.064517 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.065451 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.065961 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.067020 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.067229 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.067302 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.070538 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:51395: BlockingRecv error: recv got EOF from 127.13.195.66:51395 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 20:44:57.070878 23162 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:36881 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.195.126:36881: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:57.075147 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.071123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:48135)
0619 20:44:57.071166 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:57.071177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:57.071222 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:57.071348 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:57.071349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:57.071484 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:57.071523 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.071524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.072118 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.072120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.072617 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.072618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.072991 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.072992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.073053 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.073460 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:57.073465 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:57.075062 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 20:44:57.075192 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:57.071141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:48135 (local address 127.13.195.126:36881)
0619 20:44:57.071178 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.071180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.071209 (+ 29us) server_negotiation.cc:407] Connection header received
0619 20:44:57.071279 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.071280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.071289 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.071345 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.071593 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.072044 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.072703 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.072913 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.072991 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.075129 (+ 2138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:48135: BlockingRecv error: recv got EOF from 127.13.195.66:48135 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:57.079741 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.075486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:54989)
0619 20:44:57.075530 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:57.075541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:57.075578 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:57.075728 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:57.075729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:57.075826 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:57.075849 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.075850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.076778 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.076778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.077229 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.077230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.077703 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.077704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.077758 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.078162 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:57.078167 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:57.079697 (+ 1530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:44:57.079838 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:57.075553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:54989 (local address 127.13.195.126:36881)
0619 20:44:57.075608 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.075611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.075620 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:57.075642 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.075643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.075654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.075729 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.075939 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.076699 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.077304 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.077618 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.077688 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.079781 (+ 2093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:54989: BlockingRecv error: recv got EOF from 127.13.195.66:54989 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 20:44:57.079929 23162 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:44:57.149431 23165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:44:57.149636 23165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:44:57.149660 23165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:44:57.150807 23165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:44:57.150849 23165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:44:57.150875 23165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:44:57.152284 23165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/krb5kdc/oryx_127.13.195.67.keytab
--principal=oryx/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:37649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=46747
--tserver_master_addrs=127.13.195.126:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:44:57.152480 23165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:44:57.152662 23165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:44:57.157477 23171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:57.157487 23170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:44:57.157636 23173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:44:57.157692 23165 server_base.cc:1034] running on GCE node
I20260619 20:44:57.157920 23165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:44:57.158192 23165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:44:57.159346 23165 hybrid_clock.cc:648] HybridClock initialized: now 1781901897159326 us; error 47 us; skew 500 ppm
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901897, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:44:57.161693 23165 init.cc:377] Logged in from keytab as oryx/127.13.195.67@KRBTEST.COM (short username oryx)
I20260619 20:44:57.162142 23165 webserver.cc:460] Webserver started at http://127.13.195.67:46747/ using document root <none> and password file <none>
I20260619 20:44:57.162415 23165 fs_manager.cc:359] Metadata directory not provided
I20260619 20:44:57.162532 23165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:44:57.163861 23165 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:57.164479 23180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:57.164752 23165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:44:57.164848 23165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "1d84a612edb64b8d812b6baa43a17352"
format_stamp: "Formatted at 2026-06-19 20:44:56 on dist-test-slave-n2td"
server_key: "474304f458a8f3e95ae70efed3784f38"
server_key_iv: "eb7c7a9d0586a81442fa49f428e149ee"
server_key_version: "encryptionkey@0"
I20260619 20:44:57.165179 23165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:44:57.191672 23165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:44:57.193574 23165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:44:57.193724 23165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:44:57.193949 23165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:44:57.194245 23165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:44:57.194296 23165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:57.194339 23165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:44:57.194365 23165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:44:57.200098 23165 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:37649
I20260619 20:44:57.200176 23293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:37649 every 8 connection(s)
I20260619 20:44:57.200497 23165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:44:57.203020 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23165
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:44:57.208343 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:57.201197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:42001 (local address 127.13.195.126:36881)
0619 20:44:57.201255 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.201257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.201477 (+ 220us) server_negotiation.cc:407] Connection header received
0619 20:44:57.201945 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.201946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.201956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.202010 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.203218 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.203830 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.204750 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.205050 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.205117 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.208283 (+ 3166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:42001: BlockingRecv error: recv got EOF from 127.13.195.67:42001 (error 108)
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":29}
I20260619 20:44:57.208429 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.201170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:42001)
0619 20:44:57.201351 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:57.201393 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:57.201853 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:57.202082 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:57.202085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:57.202774 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:57.203069 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.203075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.203906 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.203907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.204692 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.204694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.205139 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.205140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.205242 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.205747 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:57.205755 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:57.208231 (+ 2476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
W20260619 20:44:57.208626 23294 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:36881 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.195.126:36881: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
I20260619 20:44:57.211294 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.203907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.13.195.126:36881)
0619 20:44:57.204019 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.204021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.204033 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:44:57.204049 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.204049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.204057 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.204129 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.204706 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.205339 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.206065 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.206288 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.206350 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.207909 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:44:57.207912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:44:57.207913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:44:57.207922 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:44:57.209816 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:57.210186 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:57.210189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:57.210189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:57.210309 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:44:57.210518 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:44:57.210519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:44:57.210519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:44:57.210813 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:44:57.210941 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 20:44:57.211072 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 20:44:57.211209 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:57.213559 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.208889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:32811)
0619 20:44:57.209101 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:57.209111 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:57.209156 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:57.209299 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:57.209300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:57.209428 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:57.209468 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.209469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.210113 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.210113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.210903 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.210905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.211595 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.211596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.211653 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.212058 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:57.212062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:57.213516 (+ 1454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198}
W20260619 20:44:57.213615 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:57.208927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:32811 (local address 127.13.195.126:36881)
0619 20:44:57.208965 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.208967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.209155 (+ 188us) server_negotiation.cc:407] Connection header received
0619 20:44:57.209215 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.209228 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.209237 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.209294 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.209536 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.210039 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.210984 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.211311 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.211377 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.213580 (+ 2203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:32811: BlockingRecv error: recv got EOF from 127.13.195.67:32811 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:57.217391 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.213828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:40011)
0619 20:44:57.213875 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:57.213886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:57.213929 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:57.214023 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:57.214024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:57.214121 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:57.214143 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.214144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.214723 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.214723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.215174 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.215176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.215456 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.215456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.215497 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.215866 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:57.215869 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:57.217329 (+ 1460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:44:57.217434 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:57.213889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:40011 (local address 127.13.195.126:36881)
0619 20:44:57.213928 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.213930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.213936 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:44:57.213973 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.213974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.213981 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.214022 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.214185 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.214663 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.215237 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.215401 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.215448 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.217398 (+ 1950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:40011: BlockingRecv error: recv got EOF from 127.13.195.67:40011 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 20:44:57.217556 23294 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:57 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:57.958612 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:57.953764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:52283)
0619 20:44:57.953903 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:57.953914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:57.953999 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:57.954139 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:57.954140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:57.954302 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:57.954352 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.954353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.955069 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.955070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.955625 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:57.955627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.955942 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.955943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:57.955985 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.956464 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:57.956469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:57.958543 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
W20260619 20:44:57.958703 23300 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:57.953869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:52283 (local address 127.13.195.126:36881)
0619 20:44:57.954017 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:57.954018 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:57.954028 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:44:57.954056 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:57.954057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:57.954068 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:57.954128 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:57.954471 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.954989 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.955695 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:57.955883 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:57.955933 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:57.958631 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:52283: BlockingRecv error: recv got EOF from 127.13.195.65:52283 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
Jun 19 20:44:58 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:58 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:58.084789 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:58.080290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:57551)
0619 20:44:58.080425 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:58.080435 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:58.080489 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:58.080665 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:58.080666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:58.080830 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:58.080895 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:58.080896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.081511 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.081511 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:58.082064 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:58.082065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.082423 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.082423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:58.082526 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:58.082978 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:58.082982 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:58.084668 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
W20260619 20:44:58.084808 23300 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:58.080382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:57551 (local address 127.13.195.126:36881)
0619 20:44:58.080450 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:58.080451 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:58.080470 (+ 19us) server_negotiation.cc:407] Connection header received
0619 20:44:58.080546 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:58.080547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:58.080564 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:58.080620 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:58.080961 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.081440 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.082154 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.082359 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.082418 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:58.084761 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:57551: BlockingRecv error: recv got EOF from 127.13.195.66:57551 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 20:44:58 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:58 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:58.222348 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:58.217956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:34155)
0619 20:44:58.218091 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:58.218107 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:58.218163 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:58.218276 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:58.218277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:58.218486 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:58.218540 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:58.218542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.219171 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.219172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:58.219739 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:58.219741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.220068 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.220069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:58.220114 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:58.220598 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:58.220603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:58.222277 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
W20260619 20:44:58.222415 23300 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:58.218022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:34155 (local address 127.13.195.126:36881)
0619 20:44:58.218102 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:58.218105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:58.218162 (+ 57us) server_negotiation.cc:407] Connection header received
0619 20:44:58.218206 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:58.218207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:58.218219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:58.218272 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:58.218617 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.219107 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.219809 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.220003 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.220054 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:58.222368 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:34155: BlockingRecv error: recv got EOF from 127.13.195.67:34155 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 20:44:58 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:58 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:58.963322 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:58.959068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:42147)
0619 20:44:58.959199 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:58.959208 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:58.959256 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:58.959398 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:58.959398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:58.959555 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:58.959604 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:58.959605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.960209 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.960210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:58.960718 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:58.960719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.961019 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.961020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:58.961063 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:58.961537 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:58.961542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:58.963263 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
W20260619 20:44:58.963486 23304 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:58.959118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:42147 (local address 127.13.195.126:36881)
0619 20:44:58.959229 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:58.959231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:58.959246 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:44:58.959308 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:58.959308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:58.959319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:58.959379 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:58.959660 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.960149 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.960779 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:58.960960 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:58.961011 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:58.963398 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:42147: BlockingRecv error: recv got EOF from 127.13.195.65:42147 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
Jun 19 20:44:59 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:59 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:59.089838 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:59.085286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:52847)
0619 20:44:59.085437 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:59.085448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:59.085505 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:59.085621 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:59.085622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:59.085782 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:59.085830 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:59.085832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.086548 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.086548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:59.087144 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:59.087146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.087573 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.087574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:59.087631 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:59.088131 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:59.088135 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:59.089777 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
W20260619 20:44:59.089957 23304 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:59.085366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:52847 (local address 127.13.195.126:36881)
0619 20:44:59.085413 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:59.085415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:59.085503 (+ 88us) server_negotiation.cc:407] Connection header received
0619 20:44:59.085550 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:59.085550 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:59.085561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:59.085616 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:59.085894 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.086387 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.087228 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.087509 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.087576 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:59.089863 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:52847: BlockingRecv error: recv got EOF from 127.13.195.66:52847 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 20:44:59 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:59 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:59.227392 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:59.222855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:60283)
0619 20:44:59.223013 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:59.223025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:59.223112 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:59.223239 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:59.223240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:59.223412 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:59.223460 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:59.223461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.224143 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.224144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:59.224668 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:59.224670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.225037 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.225038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:59.225102 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:59.225596 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:59.225600 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:59.227301 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":55,"threads_started":1}
W20260619 20:44:59.227494 23304 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:59.223051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:60283 (local address 127.13.195.126:36881)
0619 20:44:59.223118 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:59.223121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:59.223149 (+ 28us) server_negotiation.cc:407] Connection header received
0619 20:44:59.223173 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:59.223174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:59.223187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:59.223236 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:59.223555 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.224061 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.224752 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.224960 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.225028 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:59.227396 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:60283: BlockingRecv error: recv got EOF from 127.13.195.67:60283 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 20:44:59 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:44:59 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:44:59.968323 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:44:59.963818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:51217)
0619 20:44:59.963946 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 20:44:59.963965 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 20:44:59.964034 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:44:59.964174 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:44:59.964175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:44:59.964332 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:44:59.964381 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:59.964383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.964996 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.964996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:59.965780 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:44:59.965783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.966111 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.966112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:44:59.966148 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:59.966591 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:44:59.966595 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:44:59.968266 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
W20260619 20:44:59.968431 23308 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:44:59.963900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:51217 (local address 127.13.195.126:36881)
0619 20:44:59.964044 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 20:44:59.964046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:44:59.964055 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:44:59.964091 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:44:59.964092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:44:59.964102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:44:59.964162 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:44:59.964446 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.964931 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.965850 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:44:59.966042 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:44:59.966106 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:44:59.968353 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:51217: BlockingRecv error: recv got EOF from 127.13.195.65:51217 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
Jun 19 20:45:00 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:00 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:00.094772 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:00.090404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:52783)
0619 20:45:00.090590 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:00.090603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:00.090660 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:00.090789 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:00.090789 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:00.090944 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:00.090995 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:00.090996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.091609 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.091609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:00.092123 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:00.092124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.092435 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.092436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:00.092481 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:00.092921 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:00.092925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:00.094711 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
W20260619 20:45:00.094929 23308 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:00.090573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:52783 (local address 127.13.195.126:36881)
0619 20:45:00.090635 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:00.090637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:00.090661 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:45:00.090716 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:00.090717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:00.090729 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:00.090770 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:00.091055 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.091546 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.092194 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.092375 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.092425 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:00.094842 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:52783: BlockingRecv error: recv got EOF from 127.13.195.66:52783 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 20:45:00 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:00 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:00.232950 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:00.227952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:47683)
0619 20:45:00.228124 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:00.228134 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:00.228212 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:00.228362 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:00.228363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:00.228542 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:00.228590 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:00.228592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.229255 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.229257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:00.229781 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:00.229783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.230153 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.230154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:00.230213 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:00.230754 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:00.230759 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:00.232854 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
W20260619 20:45:00.233000 23308 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:00.228006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:47683 (local address 127.13.195.126:36881)
0619 20:45:00.228052 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:00.228055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:00.228195 (+ 140us) server_negotiation.cc:407] Connection header received
0619 20:45:00.228269 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:00.228270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:00.228285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:00.228355 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:00.228670 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.229163 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.229864 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.230057 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.230154 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:00.232935 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:47683: BlockingRecv error: recv got EOF from 127.13.195.67:47683 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 20:45:00 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:00 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:00.973196 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:00.968789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:52717)
0619 20:45:00.968916 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:00.968926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:00.968972 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:00.969259 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:00.969260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:00.969418 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:00.969466 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:00.969467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.970154 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.970155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:00.970729 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:00.970731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.971055 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.971055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:00.971094 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:00.971526 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:00.971530 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:00.973137 (+ 1607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
W20260619 20:45:00.973315 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:00.968985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:52717 (local address 127.13.195.126:36881)
0619 20:45:00.969134 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:00.969137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:00.969149 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:45:00.969171 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:00.969171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:00.969182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:00.969243 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:00.969579 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.970087 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.970804 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:00.970996 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:00.971048 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:00.973234 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:52717: BlockingRecv error: recv got EOF from 127.13.195.65:52717 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":103,"threads_started":1}
Jun 19 20:45:01 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:01 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:01.103729 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:01.095296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:59593)
0619 20:45:01.095459 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:01.095472 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:01.095537 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:01.095663 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:01.095663 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:01.095837 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:01.095914 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:01.095916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.096742 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.096743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:01.097270 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:01.097272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.097630 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.097631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:01.097679 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:01.101875 (+ 4196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:01.101880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:01.103669 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
W20260619 20:45:01.103840 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:01.095359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:59593 (local address 127.13.195.126:36881)
0619 20:45:01.095432 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:01.095435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:01.095536 (+ 101us) server_negotiation.cc:407] Connection header received
0619 20:45:01.095586 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:01.095587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:01.095599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:01.095648 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:01.096001 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.096677 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.097359 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.097544 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.097602 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:01.103766 (+ 6164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:59593: BlockingRecv error: recv got EOF from 127.13.195.66:59593 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 20:45:01 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:01 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:01.238548 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:01.233770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:43537)
0619 20:45:01.233919 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:01.233930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:01.233998 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:01.234171 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:01.234171 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:01.234343 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:01.234393 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:01.234394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.235096 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.235097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:01.235652 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:01.235654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.236094 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.236095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:01.236165 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:01.236640 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:01.236644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:01.238481 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":55,"threads_started":1}
W20260619 20:45:01.238605 23312 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:01.233867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:43537 (local address 127.13.195.126:36881)
0619 20:45:01.233922 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:01.233924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:01.233995 (+ 71us) server_negotiation.cc:407] Connection header received
0619 20:45:01.234057 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:01.234058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:01.234072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:01.234150 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:01.234510 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.235015 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.235749 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.236023 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.236095 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:01.238558 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:43537: BlockingRecv error: recv got EOF from 127.13.195.67:43537 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Jun 19 20:45:01 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:01 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:01.981032 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:01.973712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:47287)
0619 20:45:01.973837 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:01.973847 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:01.973893 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:01.974116 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:01.974116 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:01.974277 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:01.974328 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:01.974329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.975112 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.975113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:01.975635 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:01.975636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.976003 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.976004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:01.976043 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:01.979221 (+ 3178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:01.979227 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:01.980971 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
W20260619 20:45:01.981163 23316 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:01.973814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:47287 (local address 127.13.195.126:36881)
0619 20:45:01.973973 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:01.973975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:01.973985 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:45:01.974009 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:01.974010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:01.974020 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:01.974105 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:01.974493 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.975041 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.975714 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:01.975935 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:01.976001 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:01.981074 (+ 5073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:47287: BlockingRecv error: recv got EOF from 127.13.195.65:47287 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
Jun 19 20:45:02 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:02 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:02.109926 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:02.104170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:40075)
0619 20:45:02.104286 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:02.104298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:02.104347 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:02.104543 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:02.104544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:02.104770 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:02.104840 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:02.104842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.105809 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.105811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:02.106509 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:02.106511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.106930 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.106931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:02.106989 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:02.107526 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:02.107531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:02.109813 (+ 2282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
W20260619 20:45:02.109990 23316 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:02.104283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:40075 (local address 127.13.195.126:36881)
0619 20:45:02.104325 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:02.104328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:02.104352 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:45:02.104398 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:02.104399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:02.104409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:02.104473 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:02.104919 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.105734 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.106612 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.106846 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.106929 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:02.109924 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:40075: BlockingRecv error: recv got EOF from 127.13.195.66:40075 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 20:45:02 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:02 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:02.243960 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:02.239077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:59593)
0619 20:45:02.239255 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:02.239269 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:02.239360 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:02.239523 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:02.239524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:02.239694 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:02.239740 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:02.239741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.240407 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.240408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:02.240958 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:02.240961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.241326 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.241327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:02.241385 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:02.241874 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:02.241884 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:02.243865 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
W20260619 20:45:02.244029 23316 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:02.239186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:59593 (local address 127.13.195.126:36881)
0619 20:45:02.239260 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:02.239263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:02.239341 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:45:02.239418 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:02.239420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:02.239439 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:02.239504 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:02.239819 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.240322 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.241060 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.241245 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.241326 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:02.243946 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:59593: BlockingRecv error: recv got EOF from 127.13.195.67:59593 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 20:45:02 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:02 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:02.986187 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:02.981578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:42027)
0619 20:45:02.981714 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:02.981725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:02.981788 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:02.981942 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:02.981943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:02.982120 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:02.982187 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:02.982189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.983107 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.983108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:02.983625 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:02.983627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.983948 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.983948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:02.983985 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:02.984416 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:02.984421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:02.986131 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260619 20:45:02.986330 23320 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:02.981699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:42027 (local address 127.13.195.126:36881)
0619 20:45:02.981819 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:02.981821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:02.981829 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:45:02.981854 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:02.981854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:02.981865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:02.981924 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:02.982268 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.983034 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.983697 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:02.983888 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:02.983940 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:02.986243 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:42027: BlockingRecv error: recv got EOF from 127.13.195.65:42027 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
Jun 19 20:45:03 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:03 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:03.115209 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:03.110364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:60125)
0619 20:45:03.110639 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:03.110652 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:03.110740 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:03.110901 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:03.110902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:03.111091 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:03.111136 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:03.111138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.111796 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.111797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:03.112347 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:03.112349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.112719 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.112720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:03.112782 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:03.113296 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:03.113300 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:03.115150 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
W20260619 20:45:03.115401 23320 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:03.110465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:60125 (local address 127.13.195.126:36881)
0619 20:45:03.110639 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:03.110642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:03.110727 (+ 85us) server_negotiation.cc:407] Connection header received
0619 20:45:03.110813 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:03.110814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:03.110827 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:03.110896 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:03.111216 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.111716 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.112436 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.112636 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.112719 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:03.115301 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:60125: BlockingRecv error: recv got EOF from 127.13.195.66:60125 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
Jun 19 20:45:03 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:03 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:03.249118 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:03.244536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:34599)
0619 20:45:03.244661 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:03.244671 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:03.244754 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:03.244909 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:03.244910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:03.245130 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:03.245183 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:03.245185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.245793 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.245793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:03.246337 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:03.246338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.246668 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.246669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:03.246712 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:03.247214 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:03.247218 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:03.249046 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
W20260619 20:45:03.249176 23320 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:03.244593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:34599 (local address 127.13.195.126:36881)
0619 20:45:03.244644 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:03.244646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:03.244734 (+ 88us) server_negotiation.cc:407] Connection header received
0619 20:45:03.244817 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:03.244818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:03.244831 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:03.244885 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:03.245248 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.245725 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.246401 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.246608 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.246657 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:03.249129 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:34599: BlockingRecv error: recv got EOF from 127.13.195.67:34599 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 19 20:45:03 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:03 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:03.990880 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:03.986616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:51543)
0619 20:45:03.986759 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:03.986770 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:03.986818 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:03.986950 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:03.986951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:03.987122 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:03.987171 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:03.987173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.987785 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.987786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:03.988316 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:03.988318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.988631 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.988632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:03.988670 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:03.989087 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:03.989092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:03.990822 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
W20260619 20:45:03.991024 23324 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:03.986687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:51543 (local address 127.13.195.126:36881)
0619 20:45:03.986809 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:03.986811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:03.986826 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:45:03.986864 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:03.986865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:03.986875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:03.986935 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:03.987232 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.987722 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.988386 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:03.988575 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:03.988625 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:03.990931 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:51543: BlockingRecv error: recv got EOF from 127.13.195.65:51543 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
Jun 19 20:45:04 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:04 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:04.120277 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:04.115684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:55659)
0619 20:45:04.115862 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:04.115875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:04.115946 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:04.116097 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:04.116098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:04.116283 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:04.116332 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:04.116334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.117039 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.117040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:04.117588 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:04.117590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.117894 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.117895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:04.117932 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:04.118361 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:04.118365 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:04.120218 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":89,"threads_started":1}
W20260619 20:45:04.120424 23324 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:04.115753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:55659 (local address 127.13.195.126:36881)
0619 20:45:04.115839 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:04.115841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:04.115942 (+ 101us) server_negotiation.cc:407] Connection header received
0619 20:45:04.115998 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:04.115999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:04.116011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:04.116090 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:04.116409 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.116957 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.117660 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.117837 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.117886 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:04.120338 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:55659: BlockingRecv error: recv got EOF from 127.13.195.66:55659 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 20:45:04 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:04 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:04.254267 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:04.249581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:55673)
0619 20:45:04.249729 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:04.249742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:04.249824 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:04.249975 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:04.249976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:04.250152 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:04.250200 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:04.250202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.250927 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.250928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:04.251449 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:04.251451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.251826 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.251827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:04.251894 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:04.252360 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:04.252364 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:04.254205 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
W20260619 20:45:04.254475 23324 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:04.249667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:55673 (local address 127.13.195.126:36881)
0619 20:45:04.249733 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:04.249736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:04.249807 (+ 71us) server_negotiation.cc:407] Connection header received
0619 20:45:04.249903 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:04.249904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:04.249918 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:04.249960 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:04.250282 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.250845 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.251534 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.251742 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.251825 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:04.254333 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:55673: BlockingRecv error: recv got EOF from 127.13.195.67:55673 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 20:45:04 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:04 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:04.996342 23327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:04.991440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:50481)
0619 20:45:04.991633 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:04.991645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:04.991705 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:04.991880 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:04.991881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:04.992062 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:04.992111 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:04.992113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.993004 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.993005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:04.993535 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:04.993537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.993854 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.993854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:04.993893 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:04.994377 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:04.994382 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:04.996273 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":59,"threads_started":1}
W20260619 20:45:04.996533 23328 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:04.991635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:50481 (local address 127.13.195.126:36881)
0619 20:45:04.991757 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:04.991759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:04.991769 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:45:04.991793 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:04.991794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:04.991804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:04.991865 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:04.992186 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.992925 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.993609 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:04.993795 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:04.993846 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:04.996416 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:50481: BlockingRecv error: recv got EOF from 127.13.195.65:50481 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
Jun 19 20:45:05 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:05 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:05.125757 23329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:05.120765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:44719)
0619 20:45:05.120965 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:05.120978 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:05.121056 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:05.121209 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:05.121210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:05.121410 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:05.121456 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:05.121458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.122252 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.122253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:05.122911 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:05.122926 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.123387 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.123387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:05.123454 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:05.123949 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:05.123955 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:05.125673 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":107,"threads_started":1}
W20260619 20:45:05.125880 23328 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:05.120876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:44719 (local address 127.13.195.126:36881)
0619 20:45:05.120965 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:05.120967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:05.121048 (+ 81us) server_negotiation.cc:407] Connection header received
0619 20:45:05.121117 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:05.121118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:05.121133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:05.121204 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:05.121539 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.122166 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.123033 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.123297 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.123362 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:05.125782 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:44719: BlockingRecv error: recv got EOF from 127.13.195.66:44719 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 20:45:05 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:05 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:05.259425 23330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:05.254704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:38665)
0619 20:45:05.254870 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:05.254880 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:05.254947 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:05.255063 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:05.255063 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:05.255224 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:05.255272 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:05.255274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.255894 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.255895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:05.256403 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:05.256405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.256778 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.256780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:05.256828 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:05.257274 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:05.257279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:05.259348 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
W20260619 20:45:05.259509 23328 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:05.254754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:38665 (local address 127.13.195.126:36881)
0619 20:45:05.254817 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:05.254820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:05.254947 (+ 127us) server_negotiation.cc:407] Connection header received
0619 20:45:05.254990 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:05.254990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:05.255004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:05.255051 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:05.255327 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.255832 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.256511 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.256699 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.256766 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:05.259440 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:38665: BlockingRecv error: recv got EOF from 127.13.195.67:38665 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 20:45:05 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:05 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:06.001657 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:05.996927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:41901)
0619 20:45:05.997096 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:05.997106 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:05.997157 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:05.997325 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:05.997326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:05.997491 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:05.997542 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:05.997543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.998202 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.998203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:05.998810 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:05.998812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.999195 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.999196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:05.999259 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:05.999759 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:05.999764 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:06.001597 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
W20260619 20:45:06.001781 23332 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:05.997027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:41901 (local address 127.13.195.126:36881)
0619 20:45:05.997193 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:05.997195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:05.997205 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:45:05.997228 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:05.997229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:05.997240 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:05.997310 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:05.997618 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.998131 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.998897 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:05.999106 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:05.999193 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:06.001697 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:41901: BlockingRecv error: recv got EOF from 127.13.195.65:41901 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
Jun 19 20:45:06 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:06 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:06.135699 23333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:06.126298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:42123)
0619 20:45:06.126531 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:06.126544 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:06.126627 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:06.126790 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:06.126791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:06.126995 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:06.127042 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:06.127044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.127734 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.127736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:06.128267 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:06.128268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.128645 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.128645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:06.128701 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:06.133747 (+ 5046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:06.133752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:06.135636 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":102,"threads_started":1}
W20260619 20:45:06.135877 23332 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:06.126411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:42123 (local address 127.13.195.126:36881)
0619 20:45:06.126525 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:06.126528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:06.126604 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:45:06.126696 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:06.126697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:06.126709 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:06.126784 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:06.127129 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.127642 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.128355 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.128555 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.128644 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:06.135793 (+ 7149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:42123: BlockingRecv error: recv got EOF from 127.13.195.66:42123 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 19 20:45:06 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:06 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:06.264786 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:06.259887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:49383)
0619 20:45:06.260070 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:06.260082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:06.260172 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:06.260335 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:06.260336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:06.260522 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:06.260568 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:06.260570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.261481 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.261497 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:06.262063 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:06.262065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.262414 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.262415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:06.262513 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:06.263005 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:06.263011 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:06.264727 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
W20260619 20:45:06.264880 23332 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:06.259983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:49383 (local address 127.13.195.126:36881)
0619 20:45:06.260056 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:06.260058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:06.260151 (+ 93us) server_negotiation.cc:407] Connection header received
0619 20:45:06.260241 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:06.260242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:06.260262 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:06.260328 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:06.260646 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.261388 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.262158 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:06.262342 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:06.262399 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:06.264815 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:49383: BlockingRecv error: recv got EOF from 127.13.195.67:49383 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 20:45:07 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:07 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:07.007131 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:07.002141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:52147)
0619 20:45:07.002278 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:07.002289 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:07.002380 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:07.002645 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:07.002646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:07.002826 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:07.002877 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:07.002879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.003507 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.003508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:07.004262 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:07.004265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.004691 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.004691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:07.004731 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:07.005245 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:07.005250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:07.007060 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
W20260619 20:45:07.007263 23336 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:07.002220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:52147 (local address 127.13.195.126:36881)
0619 20:45:07.002362 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:07.002363 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:07.002373 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:45:07.002483 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:07.002483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:07.002534 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:07.002627 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:07.002946 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.003440 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.004351 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.004626 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.004679 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:07.007173 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:52147: BlockingRecv error: recv got EOF from 127.13.195.65:52147 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":52,"threads_started":1}
Jun 19 20:45:07 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:07 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:07.140846 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:07.136211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:60137)
0619 20:45:07.136361 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:07.136371 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:07.136427 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:07.136562 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:07.136563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:07.136723 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:07.136772 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:07.136773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.137494 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.137495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:07.138005 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:07.138007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.138357 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.138358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:07.138394 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:07.138904 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:07.138909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:07.140776 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":114,"threads_started":1}
W20260619 20:45:07.140992 23336 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:07.136278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:60137 (local address 127.13.195.126:36881)
0619 20:45:07.136379 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:07.136381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:07.136424 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:45:07.136483 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:07.136483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:07.136496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:07.136546 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:07.136864 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.137421 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.138096 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.138289 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.138350 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:07.140895 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:60137: BlockingRecv error: recv got EOF from 127.13.195.66:60137 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 20:45:07 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:07 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:07.270215 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:07.265407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:51623)
0619 20:45:07.265632 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:07.265659 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:07.265739 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:07.265967 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:07.265968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:07.266184 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:07.266244 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:07.266247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.266956 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.266957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:07.267472 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:07.267473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.267834 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.267835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:07.267875 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:07.268401 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:07.268406 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:07.270145 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":125,"threads_started":1}
W20260619 20:45:07.270283 23336 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:07.265569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:51623 (local address 127.13.195.126:36881)
0619 20:45:07.265627 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:07.265629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:07.265748 (+ 119us) server_negotiation.cc:407] Connection header received
0619 20:45:07.265837 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:07.265838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:07.265870 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:07.265933 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:07.266330 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.266887 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.267561 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:07.267764 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:07.267819 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:07.270235 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:51623: BlockingRecv error: recv got EOF from 127.13.195.67:51623 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 20:45:08 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:08 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:08.012437 23339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:08.007662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:56929)
0619 20:45:08.007815 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:08.007827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:08.007881 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:08.008006 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:08.008007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:08.008205 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:08.008253 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:08.008254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.008972 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.008972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:08.009510 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:08.009511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.009902 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.009902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:08.009945 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:08.010379 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:08.010384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:08.012366 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
W20260619 20:45:08.012588 23340 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:08.007709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:56929 (local address 127.13.195.126:36881)
0619 20:45:08.007841 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:08.007843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:08.007865 (+ 22us) server_negotiation.cc:407] Connection header received
0619 20:45:08.007922 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:08.007923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:08.007933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:08.008000 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:08.008314 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.008899 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.009627 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.009835 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.009889 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:08.012481 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:56929: BlockingRecv error: recv got EOF from 127.13.195.65:56929 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
Jun 19 20:45:08 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:08 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:08.145983 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:08.141336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:58595)
0619 20:45:08.141464 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:08.141475 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:08.141528 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:08.141641 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:08.141642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:08.141801 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:08.141847 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:08.141848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.142592 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.142593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:08.143139 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:08.143141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.143491 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.143492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:08.143550 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:08.144038 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:08.144043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:08.145912 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
W20260619 20:45:08.146049 23340 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:08.141415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:58595 (local address 127.13.195.126:36881)
0619 20:45:08.141506 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:08.141508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:08.141516 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:45:08.141586 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:08.141587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:08.141597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:08.141636 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:08.141942 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.142475 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.143213 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.143397 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.143450 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:08.146003 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:58595: BlockingRecv error: recv got EOF from 127.13.195.66:58595 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 20:45:08 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:08 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:08.275393 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:08.270690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:57837)
0619 20:45:08.270804 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:08.270821 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:08.270884 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:08.271017 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:08.271018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:08.271199 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:08.271245 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:08.271246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.271877 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.271879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:08.272418 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:08.272419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.272758 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.272759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:08.272798 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:08.273266 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:08.273271 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:08.275322 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
W20260619 20:45:08.275460 23340 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:08.270755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:57837 (local address 127.13.195.126:36881)
0619 20:45:08.270837 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:08.270839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:08.270871 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:45:08.270942 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:08.270943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:08.270955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:08.270996 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:08.271313 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.271810 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.272504 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:08.272691 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:08.272746 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:08.275413 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:57837: BlockingRecv error: recv got EOF from 127.13.195.67:57837 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 20:45:09 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:09 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:09.017709 23343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:09.012978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:58621)
0619 20:45:09.013137 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:09.013147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:09.013205 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:09.013516 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:09.013517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:09.013775 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:09.013834 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:09.013837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.014539 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.014540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:09.015064 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:09.015066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.015425 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.015426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:09.015465 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:09.015895 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:09.015899 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:09.017650 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
W20260619 20:45:09.017859 23344 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:09.013080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:58621 (local address 127.13.195.126:36881)
0619 20:45:09.013208 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:09.013211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:09.013223 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:45:09.013330 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:09.013331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:09.013345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:09.013484 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:09.013925 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.014478 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.015140 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.015332 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.015420 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:09.017765 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:58621: BlockingRecv error: recv got EOF from 127.13.195.65:58621 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":50,"threads_started":1}
Jun 19 20:45:09 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:09 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:09.151067 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:09.146479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:46715)
0619 20:45:09.146629 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:09.146643 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:09.146709 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:09.146824 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:09.146824 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:09.147005 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:09.147050 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:09.147051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.147693 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.147694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:09.148236 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:09.148237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.148551 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.148552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:09.148591 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:09.149100 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:09.149105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:09.150970 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
W20260619 20:45:09.151130 23344 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:09.146540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:46715 (local address 127.13.195.126:36881)
0619 20:45:09.146620 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:09.146622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:09.146708 (+ 86us) server_negotiation.cc:407] Connection header received
0619 20:45:09.146756 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:09.146757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:09.146768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:09.146811 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:09.147109 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.147606 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.148303 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.148488 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.148542 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:09.151054 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:46715: BlockingRecv error: recv got EOF from 127.13.195.66:46715 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 20:45:09 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:09 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:09.280462 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:09.275889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:51695)
0619 20:45:09.276019 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:09.276030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:09.276101 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:09.276214 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:09.276215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:09.276429 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:09.276474 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:09.276475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.277100 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.277101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:09.277667 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:09.277669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.277993 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.277994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:09.278048 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:09.278551 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:09.278556 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:09.280361 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
W20260619 20:45:09.280515 23344 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:09.275969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:51695 (local address 127.13.195.126:36881)
0619 20:45:09.276021 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:09.276023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:09.276101 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:45:09.276142 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:09.276143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:09.276154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:09.276210 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:09.276533 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.277025 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.277736 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:09.277910 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:09.277960 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:09.280451 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:51695: BlockingRecv error: recv got EOF from 127.13.195.67:51695 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 20:45:10 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:10 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:10.022817 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:10.018177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:47703)
0619 20:45:10.018320 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:10.018332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:10.018454 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:10.018649 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:10.018651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:10.018830 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:10.018878 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:10.018879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.019560 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.019561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:10.020120 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:10.020122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.020441 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.020441 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:10.020482 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:10.020941 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:10.020946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:10.022744 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
W20260619 20:45:10.022919 23348 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:10.018245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:47703 (local address 127.13.195.126:36881)
0619 20:45:10.018399 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:10.018402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:10.018421 (+ 19us) server_negotiation.cc:407] Connection header received
0619 20:45:10.018515 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:10.018516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:10.018530 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:10.018610 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:10.018959 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.019448 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.020194 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.020377 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.020426 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:10.022844 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:47703: BlockingRecv error: recv got EOF from 127.13.195.65:47703 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
Jun 19 20:45:10 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:10 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:10.156114 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:10.151519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:35675)
0619 20:45:10.151659 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:10.151672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:10.151739 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:10.151917 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:10.151918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:10.152099 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:10.152143 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:10.152145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.152791 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.152792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:10.153334 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:10.153336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.153689 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.153690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:10.153748 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:10.154247 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:10.154251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:10.156044 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
W20260619 20:45:10.156252 23348 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:10.151642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:35675 (local address 127.13.195.126:36881)
0619 20:45:10.151694 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:10.151696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:10.151720 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:45:10.151820 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:10.151822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:10.151840 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:10.151895 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:10.152222 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.152711 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.153422 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.153617 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.153691 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:10.156171 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:35675: BlockingRecv error: recv got EOF from 127.13.195.66:35675 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 20:45:10 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:10 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:10.285737 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:10.281071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:60363)
0619 20:45:10.281237 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:10.281248 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:10.281341 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:10.281488 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:10.281489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:10.281679 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:10.281722 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:10.281723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.282396 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.282397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:10.282957 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:10.282958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.283315 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.283316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:10.283372 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:10.283880 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:10.283885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:10.285625 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
W20260619 20:45:10.285778 23348 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:10.281130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:60363 (local address 127.13.195.126:36881)
0619 20:45:10.281206 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:10.281208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:10.281323 (+ 115us) server_negotiation.cc:407] Connection header received
0619 20:45:10.281397 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:10.281398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:10.281412 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:10.281483 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:10.281812 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.282305 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.283050 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:10.283243 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:10.283318 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:10.285713 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:60363: BlockingRecv error: recv got EOF from 127.13.195.67:60363 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 20:45:11 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:11 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:11.028062 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:11.023390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:44449)
0619 20:45:11.023572 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:11.023582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:11.023653 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:11.023834 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:11.023835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:11.024067 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:11.024128 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:11.024131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.024764 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.024765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:11.025280 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:11.025281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.025627 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.025628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:11.025665 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:11.026134 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:11.026139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:11.027962 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
W20260619 20:45:11.028131 23352 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:11.023526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:44449 (local address 127.13.195.126:36881)
0619 20:45:11.023664 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:11.023666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:11.023676 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:45:11.023732 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:11.023732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:11.023746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:11.023812 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:11.024202 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.024696 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.025349 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.025567 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.025618 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:11.028046 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:44449: BlockingRecv error: recv got EOF from 127.13.195.65:44449 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
Jun 19 20:45:11 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:11 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:11.164445 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:11.156575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:37217)
0619 20:45:11.156776 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:11.156788 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:11.156880 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:11.157056 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:11.157057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:11.157247 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:11.157295 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:11.157296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.158038 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.158039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:11.158652 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:11.158654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.159032 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.159033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:11.159097 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:11.162341 (+ 3244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:11.162345 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:11.164343 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":118,"threads_started":1}
W20260619 20:45:11.164602 23352 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:11.156671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:37217 (local address 127.13.195.126:36881)
0619 20:45:11.156762 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:11.156765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:11.156859 (+ 94us) server_negotiation.cc:407] Connection header received
0619 20:45:11.156961 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:11.156962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:11.156974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:11.157052 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:11.157376 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.157944 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.158731 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.158943 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.159032 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:11.164517 (+ 5485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:37217: BlockingRecv error: recv got EOF from 127.13.195.66:37217 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 20:45:11 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:11 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.67@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:11.290936 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:11.286253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.67:37061)
0619 20:45:11.286400 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:11.286409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:11.286491 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:11.286746 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:11.286748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:11.286921 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:11.286970 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:11.286971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.287602 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.287603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:11.288135 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:11.288137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.288460 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.288461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:11.288506 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:11.288940 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:11.288944 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:11.290872 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
W20260619 20:45:11.291115 23352 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:11.286297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:37061 (local address 127.13.195.126:36881)
0619 20:45:11.286364 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:11.286366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:11.286492 (+ 126us) server_negotiation.cc:407] Connection header received
0619 20:45:11.286549 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:11.286549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:11.286562 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:11.286618 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:11.287044 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.287534 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.288208 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:11.288382 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:11.288431 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:11.291028 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.67:37061: BlockingRecv error: recv got EOF from 127.13.195.67:37061 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 20:45:12 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:12 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.65@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:12.033989 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.028883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.65:52853)
0619 20:45:12.029046 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:12.029057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:12.029124 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:12.029284 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:12.029284 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:12.029460 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:12.029509 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.029511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.030268 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.030269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.031056 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.031059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.031439 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.031440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.031478 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.031971 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:12.031976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:12.033920 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
W20260619 20:45:12.034103 23356 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:12.028905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:52853 (local address 127.13.195.126:36881)
0619 20:45:12.029031 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.029033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.029123 (+ 90us) server_negotiation.cc:407] Connection header received
0619 20:45:12.029188 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.029189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.029208 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.029262 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.029587 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.030101 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.031137 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.031333 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.031441 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.034018 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.65:52853: BlockingRecv error: recv got EOF from 127.13.195.65:52853 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
Jun 19 20:45:12 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:45:12 dist-test-slave-n2td krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.195.66@KRBTEST.COM for oryx/127.13.195.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:45:12.170941 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.164985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36881 (local address 127.13.195.66:38425)
0619 20:45:12.165130 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:12.165142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:12.165209 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:12.165352 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:12.165352 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:12.165535 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:12.165580 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.165582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.166529 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.166531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.167613 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.167618 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.168119 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.168120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.168191 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.168744 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:12.168750 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:12.170835 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.195.126:36881: Server oryx/127.13.195.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
W20260619 20:45:12.171133 23356 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:12.165055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:38425 (local address 127.13.195.126:36881)
0619 20:45:12.165120 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.165122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.165189 (+ 67us) server_negotiation.cc:407] Connection header received
0619 20:45:12.165275 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.165276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.165289 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.165345 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.165661 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.166402 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.167713 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.168024 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.168105 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.170959 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.195.66:38425: BlockingRecv error: recv got EOF from 127.13.195.66:38425 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:45:12.203585 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 22881
I20260619 20:45:12.208168 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23033
I20260619 20:45:12.212675 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23165
I20260619 20:45:12.216799 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 22811
2026-06-19T20:45:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17473 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T20:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:45:12Z Disabled control of system clock
I20260619 20:45:12.244639 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36069
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:36069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 20:45:12.324707 23361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:12.324916 23361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:12.324936 23361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:12.326145 23361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:45:12.326189 23361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:12.326201 23361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:12.326215 23361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:45:12.326256 23361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:45:12.327636 23361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:36069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:12.327905 23361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:12.328231 23361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:12.333464 23367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.333472 23361 server_base.cc:1034] running on GCE node
W20260619 20:45:12.333513 23366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:12.333580 23369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.334062 23361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:12.334359 23361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:12.335512 23361 hybrid_clock.cc:648] HybridClock initialized: now 1781901912335482 us; error 45 us; skew 500 ppm
I20260619 20:45:12.336825 23361 webserver.cc:460] Webserver started at http://127.13.195.126:36393/ using document root <none> and password file <none>
I20260619 20:45:12.337067 23361 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:12.337136 23361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:12.337282 23361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:12.338387 23361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "373f17ba5b1a492e99e401d29d155041"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "5bf368572da47397eb1b608493a76f63"
server_key_iv: "8b7b4afb5ec3ff3db1299216b05b3ed7"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.338821 23361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "373f17ba5b1a492e99e401d29d155041"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "5bf368572da47397eb1b608493a76f63"
server_key_iv: "8b7b4afb5ec3ff3db1299216b05b3ed7"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.340298 23361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:45:12.341145 23375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.341333 23361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:45:12.341464 23361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "373f17ba5b1a492e99e401d29d155041"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "5bf368572da47397eb1b608493a76f63"
server_key_iv: "8b7b4afb5ec3ff3db1299216b05b3ed7"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.341542 23361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:12.384474 23361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:12.384847 23361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:12.385017 23361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:12.389653 23361 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:36069
I20260619 20:45:12.389688 23427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:36069 every 8 connection(s)
I20260619 20:45:12.390048 23361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:45:12.390789 23428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:45:12.393379 23428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041: Bootstrap starting.
I20260619 20:45:12.394001 23428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041: Neither blocks nor log segments found. Creating new log.
I20260619 20:45:12.394243 23428 log.cc:824] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041: Log is configured to *not* fsync() on all Append() calls
I20260619 20:45:12.394981 23428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041: No bootstrap required, opened a new log
I20260619 20:45:12.396507 23428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373f17ba5b1a492e99e401d29d155041" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36069 } }
I20260619 20:45:12.396646 23428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:45:12.396667 23428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373f17ba5b1a492e99e401d29d155041, State: Initialized, Role: FOLLOWER
I20260619 20:45:12.396770 23428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373f17ba5b1a492e99e401d29d155041" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36069 } }
I20260619 20:45:12.396857 23428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:45:12.396900 23428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:45:12.396950 23428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:45:12.397478 23428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373f17ba5b1a492e99e401d29d155041" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36069 } }
I20260619 20:45:12.397609 23428 leader_election.cc:302] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 373f17ba5b1a492e99e401d29d155041; no voters:
I20260619 20:45:12.397789 23428 leader_election.cc:288] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:45:12.397898 23431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:45:12.398058 23428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:45:12.398132 23431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [term 1 LEADER]: Becoming Leader. State: Replica: 373f17ba5b1a492e99e401d29d155041, State: Running, Role: LEADER
I20260619 20:45:12.398248 23431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373f17ba5b1a492e99e401d29d155041" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36069 } }
I20260619 20:45:12.398651 23432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "373f17ba5b1a492e99e401d29d155041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373f17ba5b1a492e99e401d29d155041" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36069 } } }
I20260619 20:45:12.398730 23432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [sys.catalog]: This master's current role is: LEADER
I20260619 20:45:12.398809 23433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 373f17ba5b1a492e99e401d29d155041. Latest consensus state: current_term: 1 leader_uuid: "373f17ba5b1a492e99e401d29d155041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "373f17ba5b1a492e99e401d29d155041" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36069 } } }
I20260619 20:45:12.398912 23440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:45:12.398927 23433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041 [sys.catalog]: This master's current role is: LEADER
I20260619 20:45:12.399499 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23361
I20260619 20:45:12.399606 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:45:12.399852 14093 external_mini_cluster.cc:1428] Setting key 71d9427d078e59bdc1314aaeb98d4549
I20260619 20:45:12.400266 23440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:45:12.401948 23440 catalog_manager.cc:1347] Generated new cluster ID: 9e145ae0994c4189be532fac32dfa45d
I20260619 20:45:12.401999 23440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:45:12.406054 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.400343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.13.195.126:36069)
0619 20:45:12.400579 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.400588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.400800 (+ 212us) server_negotiation.cc:407] Connection header received
0619 20:45:12.400988 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.400996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.401084 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.401226 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.401432 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.402801 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.403871 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.404522 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.404649 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.405333 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:12.405336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:12.405337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:12.405552 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:12.405594 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:12.405608 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:12.405680 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:12.405746 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 20:45:12.405825 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":66,"threads_started":1}
I20260619 20:45:12.415454 23440 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:45:12.416137 23440 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:45:12.423341 23440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 373f17ba5b1a492e99e401d29d155041: Generated new TSK 0
I20260619 20:45:12.423522 23440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:45:12.427369 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36069
--builtin_ntp_servers=127.13.195.84:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 20:45:12.510718 23452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:12.510977 23452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:12.511014 23452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:12.512279 23452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:12.512341 23452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:12.512387 23452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:45:12.513754 23452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:12.514021 23452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:12.514322 23452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:12.519359 23458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:12.519375 23457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.519532 23452 server_base.cc:1034] running on GCE node
W20260619 20:45:12.519629 23460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.519850 23452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:12.520115 23452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:12.521281 23452 hybrid_clock.cc:648] HybridClock initialized: now 1781901912521267 us; error 30 us; skew 500 ppm
I20260619 20:45:12.522710 23452 webserver.cc:460] Webserver started at http://127.13.195.65:46113/ using document root <none> and password file <none>
I20260619 20:45:12.522957 23452 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:12.523039 23452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:12.523204 23452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:12.524343 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "ba2cfea519ad411ea8573db156800e3f"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "6e9513fa17302da893f60e27eeb0e3e4"
server_key_iv: "f84468d056c4ad79ad75174324018178"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.524710 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "ba2cfea519ad411ea8573db156800e3f"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "6e9513fa17302da893f60e27eeb0e3e4"
server_key_iv: "f84468d056c4ad79ad75174324018178"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.526080 23452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:45:12.526962 23466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.527215 23452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:45:12.527330 23452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "ba2cfea519ad411ea8573db156800e3f"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "6e9513fa17302da893f60e27eeb0e3e4"
server_key_iv: "f84468d056c4ad79ad75174324018178"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.527438 23452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:12.544399 23452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:12.544734 23452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:12.544896 23452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:12.545162 23452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:12.545601 23452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:12.545660 23452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.545706 23452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:12.545738 23452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.552313 23452 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:40059
I20260619 20:45:12.552356 23579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:40059 every 8 connection(s)
I20260619 20:45:12.552768 23452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:45:12.558368 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.553572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:40947 (local address 127.13.195.126:36069)
0619 20:45:12.553665 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.553668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.553881 (+ 213us) server_negotiation.cc:407] Connection header received
0619 20:45:12.554519 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.554521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.554537 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.554602 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.555285 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.555801 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.556824 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.557045 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.557135 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.558058 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:12.558060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:12.558060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:12.558065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:12.558080 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:12.558094 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:12.558159 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:12.558256 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:45:12.558288 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:45:12.558650 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.553543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36069 (local address 127.13.195.65:40947)
0619 20:45:12.553761 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:12.553796 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:12.554325 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:12.554676 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:12.554680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:12.554690 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:12.555189 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.555193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.555908 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.555909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.556717 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.556719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.557146 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.557148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.557289 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.557884 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:45:12.557885 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:12.557914 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:45:12.557937 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:12.558161 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:12.558168 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:12.558171 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:45:12.558260 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:45:12.558366 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260619 20:45:12.559249 23580 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36069
I20260619 20:45:12.559353 23580 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:12.559583 23580 heartbeater.cc:511] Master 127.13.195.126:36069 requested a full tablet report, sending...
I20260619 20:45:12.560163 23392 ts_manager.cc:194] Registered new tserver with Master: ba2cfea519ad411ea8573db156800e3f (127.13.195.65:40059)
I20260619 20:45:12.560838 23392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.65:40947
I20260619 20:45:12.562517 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23452
I20260619 20:45:12.562599 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:45:12.562795 14093 external_mini_cluster.cc:1428] Setting key 44bf39d03d1a0782b9dc240dc49ac9ce
I20260619 20:45:12.563621 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36069
--builtin_ntp_servers=127.13.195.84:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 20:45:12.645888 23583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:12.646127 23583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:12.646167 23583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:12.647373 23583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:12.647440 23583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:12.647501 23583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:45:12.649359 23583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:12.649668 23583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:12.650025 23583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:12.655704 23588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:12.655736 23589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:12.655707 23591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.655938 23583 server_base.cc:1034] running on GCE node
I20260619 20:45:12.656133 23583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:12.656371 23583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:12.657516 23583 hybrid_clock.cc:648] HybridClock initialized: now 1781901912657506 us; error 29 us; skew 500 ppm
I20260619 20:45:12.658815 23583 webserver.cc:460] Webserver started at http://127.13.195.66:45565/ using document root <none> and password file <none>
I20260619 20:45:12.659057 23583 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:12.659106 23583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:12.659252 23583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:12.660328 23583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "97db414fafec4736a1b9464359044f45"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "b803bb98458825e51c747201676ce0f9"
server_key_iv: "1d6154a0a87d8feef49afd5c453690f4"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.660688 23583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "97db414fafec4736a1b9464359044f45"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "b803bb98458825e51c747201676ce0f9"
server_key_iv: "1d6154a0a87d8feef49afd5c453690f4"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.662027 23583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:45:12.662873 23597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.663058 23583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:45:12.663130 23583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "97db414fafec4736a1b9464359044f45"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "b803bb98458825e51c747201676ce0f9"
server_key_iv: "1d6154a0a87d8feef49afd5c453690f4"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.663216 23583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:12.683619 23583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:12.683959 23583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:12.684121 23583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:12.684379 23583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:12.684733 23583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:12.684789 23583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.684825 23583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:12.684851 23583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.690853 23583 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:42005
I20260619 20:45:12.690903 23710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:42005 every 8 connection(s)
I20260619 20:45:12.691227 23583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:45:12.696307 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.692009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:45447 (local address 127.13.195.126:36069)
0619 20:45:12.692084 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.692086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.692339 (+ 253us) server_negotiation.cc:407] Connection header received
0619 20:45:12.692807 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.692808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.692818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.692873 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.693528 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.694013 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.695022 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.695211 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.695268 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.696045 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:12.696047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:12.696047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:12.696051 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:12.696067 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:12.696071 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:12.696132 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:12.696210 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 20:45:12.696236 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 20:45:12.696465 23713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.692025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36069 (local address 127.13.195.66:45447)
0619 20:45:12.692214 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:12.692247 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:12.692720 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:12.692940 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:12.692943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:12.692952 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:12.693443 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.693447 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.694091 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.694091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.694942 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.694944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.695286 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.695287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.695396 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.695892 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:45:12.695893 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:12.695920 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:45:12.695943 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:12.696140 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:12.696144 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:12.696146 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:45:12.696199 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:45:12.696277 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260619 20:45:12.696969 23711 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36069
I20260619 20:45:12.697082 23711 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:12.697259 23711 heartbeater.cc:511] Master 127.13.195.126:36069 requested a full tablet report, sending...
I20260619 20:45:12.697701 23392 ts_manager.cc:194] Registered new tserver with Master: 97db414fafec4736a1b9464359044f45 (127.13.195.66:42005)
I20260619 20:45:12.698064 23392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.66:45447
I20260619 20:45:12.698624 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23583
I20260619 20:45:12.698704 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:45:12.698952 14093 external_mini_cluster.cc:1428] Setting key 922991b26fa20fcf365e582b4d46cad3
I20260619 20:45:12.699769 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36069
--builtin_ntp_servers=127.13.195.84:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 20:45:12.780548 23714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:12.780750 23714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:12.780769 23714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:12.781975 23714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:12.782027 23714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:12.782052 23714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:45:12.783442 23714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:12.783689 23714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:12.784003 23714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:12.789124 23720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.789202 23714 server_base.cc:1034] running on GCE node
W20260619 20:45:12.789098 23722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:12.789117 23719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.789539 23714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:12.789795 23714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:12.790966 23714 hybrid_clock.cc:648] HybridClock initialized: now 1781901912790937 us; error 38 us; skew 500 ppm
I20260619 20:45:12.792384 23714 webserver.cc:460] Webserver started at http://127.13.195.67:32903/ using document root <none> and password file <none>
I20260619 20:45:12.792650 23714 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:12.792701 23714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:12.792876 23714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:12.794023 23714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "4c1574a87b8f4b45b767d50d3e36a1f1"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "b6405ce2cad180e1b415552190e49601"
server_key_iv: "6e2ab3b7f18ee43045abb9f38d5461ff"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.794397 23714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "4c1574a87b8f4b45b767d50d3e36a1f1"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "b6405ce2cad180e1b415552190e49601"
server_key_iv: "6e2ab3b7f18ee43045abb9f38d5461ff"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.795745 23714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:12.796531 23728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.796810 23714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:12.796914 23714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "4c1574a87b8f4b45b767d50d3e36a1f1"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "b6405ce2cad180e1b415552190e49601"
server_key_iv: "6e2ab3b7f18ee43045abb9f38d5461ff"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.796988 23714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:12.817809 23714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:12.818154 23714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:12.818311 23714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:12.818626 23714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:12.818975 23714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:12.819037 23714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.819080 23714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:12.819108 23714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.824872 23714 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:45579
I20260619 20:45:12.824910 23841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:45579 every 8 connection(s)
I20260619 20:45:12.825285 23714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:45:12.830698 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.826070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:60477 (local address 127.13.195.126:36069)
0619 20:45:12.826116 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.826118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.826347 (+ 229us) server_negotiation.cc:407] Connection header received
0619 20:45:12.826913 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.826914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.826925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.826987 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.827655 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.828183 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.829183 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.829384 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.829464 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.830372 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:12.830373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:12.830373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:12.830378 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:12.830422 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:12.830484 (+ 62us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:12.830541 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:12.830610 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 20:45:12.830636 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:45:12.830955 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.826049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:36069 (local address 127.13.195.67:60477)
0619 20:45:12.826223 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:12.826258 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:12.826816 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:12.827042 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:12.827045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:12.827054 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:12.827545 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.827548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.828286 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.828287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.829091 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:12.829093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:12.829476 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:12.829477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:12.829592 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:12.830212 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:45:12.830213 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:12.830242 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:45:12.830266 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:12.830553 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:12.830557 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:12.830560 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:45:12.830611 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:45:12.830694 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 20:45:12.831547 23842 heartbeater.cc:347] Connected to a master server at 127.13.195.126:36069
I20260619 20:45:12.831656 23842 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:12.831844 23842 heartbeater.cc:511] Master 127.13.195.126:36069 requested a full tablet report, sending...
I20260619 20:45:12.832278 23392 ts_manager.cc:194] Registered new tserver with Master: 4c1574a87b8f4b45b767d50d3e36a1f1 (127.13.195.67:45579)
I20260619 20:45:12.832685 23392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.67:60477
I20260619 20:45:12.834686 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23714
I20260619 20:45:12.834779 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:45:12.835006 14093 external_mini_cluster.cc:1428] Setting key 9c6a76c8e0fbaacb9e3f7f0bbacebc2b
I20260619 20:45:12.835889 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:45:12.837845 23450 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:12.837286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57288 (local address 127.13.195.126:36069)
0619 20:45:12.837349 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.837351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.837510 (+ 159us) server_negotiation.cc:407] Connection header received
0619 20:45:12.837557 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.837558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.837569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.837617 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.837764 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57288: BlockingRecv error: recv got EOF from 127.0.0.1:57288 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:45:12.837935 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23452
I20260619 20:45:12.843276 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23583
I20260619 20:45:12.847949 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23714
I20260619 20:45:12.852365 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23361
2026-06-19T20:45:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (634 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T20:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:45:12Z Disabled control of system clock
I20260619 20:45:12.878654 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:43077
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:43077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:45:12.958841 23853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:12.959040 23853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:12.959060 23853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:12.960209 23853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:45:12.960253 23853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:12.960265 23853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:12.960285 23853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:45:12.960299 23853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:45:12.961627 23853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:43077
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:43077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:12.961882 23853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:12.962175 23853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:12.966770 23861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:12.966774 23859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:12.966805 23858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:12.967097 23853 server_base.cc:1034] running on GCE node
I20260619 20:45:12.967340 23853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:12.967651 23853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:12.968797 23853 hybrid_clock.cc:648] HybridClock initialized: now 1781901912968785 us; error 27 us; skew 500 ppm
I20260619 20:45:12.969970 23853 webserver.cc:460] Webserver started at http://127.13.195.126:39735/ using document root <none> and password file <none>
I20260619 20:45:12.970201 23853 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:12.970247 23853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:12.970391 23853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:12.971486 23853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "c145768cae2f4b4e886b4ca9193d86c5"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "cd479e66e3db6474fb2c4c9b785ef6d5"
server_key_iv: "ca6a675ded858ba32ad006d133edcb3b"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.971866 23853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "c145768cae2f4b4e886b4ca9193d86c5"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "cd479e66e3db6474fb2c4c9b785ef6d5"
server_key_iv: "ca6a675ded858ba32ad006d133edcb3b"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.973176 23853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:45:12.973950 23867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:12.974159 23853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:12.974253 23853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "c145768cae2f4b4e886b4ca9193d86c5"
format_stamp: "Formatted at 2026-06-19 20:45:12 on dist-test-slave-n2td"
server_key: "cd479e66e3db6474fb2c4c9b785ef6d5"
server_key_iv: "ca6a675ded858ba32ad006d133edcb3b"
server_key_version: "encryptionkey@0"
I20260619 20:45:12.974335 23853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:12.977591 23853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:12.977852 23853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:12.978005 23853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:12.981868 23853 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:43077
I20260619 20:45:12.981899 23919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:43077 every 8 connection(s)
I20260619 20:45:12.982231 23853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:45:12.982656 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23853
I20260619 20:45:12.982761 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:45:12.982952 14093 external_mini_cluster.cc:1428] Setting key e76db44cc9f14e5ed10666b15274dcff
I20260619 20:45:12.982978 23920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:45:12.985682 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:12.983679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.13.195.126:43077)
0619 20:45:12.983853 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:12.983866 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:12.983889 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:45:12.984344 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:12.984352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:12.984390 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:12.984505 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:12.985206 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:12.985209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:12.985210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:12.985213 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:12.985251 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:12.985261 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:12.985316 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:12.985404 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 20:45:12.985492 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260619 20:45:12.986421 23920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5: Bootstrap starting.
I20260619 20:45:12.987067 23920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5: Neither blocks nor log segments found. Creating new log.
I20260619 20:45:12.987350 23920 log.cc:824] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:45:12.988044 23920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5: No bootstrap required, opened a new log
I20260619 20:45:12.989389 23920 raft_consensus.cc:348] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c145768cae2f4b4e886b4ca9193d86c5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 43077 } }
I20260619 20:45:12.989553 23920 raft_consensus.cc:374] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:45:12.989578 23920 raft_consensus.cc:729] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c145768cae2f4b4e886b4ca9193d86c5, State: Initialized, Role: FOLLOWER
I20260619 20:45:12.989712 23920 consensus_queue.cc:260] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c145768cae2f4b4e886b4ca9193d86c5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 43077 } }
I20260619 20:45:12.989786 23920 raft_consensus.cc:388] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:45:12.989831 23920 raft_consensus.cc:482] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:45:12.989880 23920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:45:12.990423 23920 raft_consensus.cc:504] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c145768cae2f4b4e886b4ca9193d86c5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 43077 } }
I20260619 20:45:12.990602 23920 leader_election.cc:302] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c145768cae2f4b4e886b4ca9193d86c5; no voters:
I20260619 20:45:12.990768 23920 leader_election.cc:288] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:45:12.991007 23925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:45:12.991035 23920 sys_catalog.cc:564] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:45:12.991145 23925 raft_consensus.cc:686] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [term 1 LEADER]: Becoming Leader. State: Replica: c145768cae2f4b4e886b4ca9193d86c5, State: Running, Role: LEADER
I20260619 20:45:12.991247 23925 consensus_queue.cc:237] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c145768cae2f4b4e886b4ca9193d86c5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 43077 } }
I20260619 20:45:12.991640 23927 sys_catalog.cc:455] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c145768cae2f4b4e886b4ca9193d86c5. Latest consensus state: current_term: 1 leader_uuid: "c145768cae2f4b4e886b4ca9193d86c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c145768cae2f4b4e886b4ca9193d86c5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 43077 } } }
I20260619 20:45:12.991637 23926 sys_catalog.cc:455] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c145768cae2f4b4e886b4ca9193d86c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c145768cae2f4b4e886b4ca9193d86c5" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 43077 } } }
I20260619 20:45:12.991750 23927 sys_catalog.cc:458] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:45:12.991763 23926 sys_catalog.cc:458] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:45:12.991966 23934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:45:12.992457 23934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:45:12.993993 23934 catalog_manager.cc:1347] Generated new cluster ID: dba34f1b28be4866a5ca9f75016b0dbd
I20260619 20:45:12.994046 23934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:45:13.012002 23934 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:45:13.012212 23934 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:45:13.024945 23934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c145768cae2f4b4e886b4ca9193d86c5: Generated new TSK 0
I20260619 20:45:13.025169 23934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:45:13.032652 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:43077
--builtin_ntp_servers=127.13.195.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:45:13.114173 23944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:13.114442 23944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:13.114481 23944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:13.115698 23944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:13.115741 23944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:13.115765 23944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:45:13.117121 23944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:13.117394 23944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:13.117722 23944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:13.122732 23950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:13.122732 23952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:13.122884 23944 server_base.cc:1034] running on GCE node
W20260619 20:45:13.122825 23949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:13.123178 23944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:13.123375 23944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:13.124542 23944 hybrid_clock.cc:648] HybridClock initialized: now 1781901913124527 us; error 31 us; skew 500 ppm
I20260619 20:45:13.125892 23944 webserver.cc:460] Webserver started at http://127.13.195.65:36435/ using document root <none> and password file <none>
I20260619 20:45:13.126130 23944 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:13.126178 23944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:13.126346 23944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:13.127456 23944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "c3e3e685b05b47acaa51804e095001c1"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "cc1c8c636fe46d3eb79cd52a013e6a06"
server_key_iv: "99906f98b9f42d7da1bd2b45c585b7d6"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.127820 23944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "c3e3e685b05b47acaa51804e095001c1"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "cc1c8c636fe46d3eb79cd52a013e6a06"
server_key_iv: "99906f98b9f42d7da1bd2b45c585b7d6"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.129223 23944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:45:13.129980 23958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.130157 23944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:45:13.130268 23944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "c3e3e685b05b47acaa51804e095001c1"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "cc1c8c636fe46d3eb79cd52a013e6a06"
server_key_iv: "99906f98b9f42d7da1bd2b45c585b7d6"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.130344 23944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:13.133651 23944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:13.133922 23944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:13.134068 23944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:13.134271 23944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:13.134625 23944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:13.134680 23944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.134722 23944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:13.134780 23944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.140457 23944 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:37957
I20260619 20:45:13.140518 24071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:37957 every 8 connection(s)
I20260619 20:45:13.140833 23944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:45:13.143774 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:13.141601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:38597 (local address 127.13.195.126:43077)
0619 20:45:13.141643 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:13.141645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:13.141871 (+ 226us) server_negotiation.cc:407] Connection header received
0619 20:45:13.142306 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:13.142307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:13.142312 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:13.142363 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:13.143562 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:13.143564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:13.143564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:13.143568 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:13.143580 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:13.143584 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:13.143635 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:13.143709 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 20:45:13.143721 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:45:13.143975 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:13.141579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:43077 (local address 127.13.195.65:38597)
0619 20:45:13.141752 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:13.141784 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:13.142223 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:13.142416 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:13.142419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:13.142471 (+ 52us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:13.143432 (+ 961us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:45:13.143432 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:13.143459 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:45:13.143486 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:13.143636 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:13.143638 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:13.143641 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:45:13.143692 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:45:13.143737 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260619 20:45:13.144481 24072 heartbeater.cc:347] Connected to a master server at 127.13.195.126:43077
I20260619 20:45:13.144587 24072 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:13.144738 24072 heartbeater.cc:511] Master 127.13.195.126:43077 requested a full tablet report, sending...
I20260619 20:45:13.145309 23884 ts_manager.cc:194] Registered new tserver with Master: c3e3e685b05b47acaa51804e095001c1 (127.13.195.65:37957)
I20260619 20:45:13.147816 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 23944
I20260619 20:45:13.147900 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:45:13.148125 14093 external_mini_cluster.cc:1428] Setting key e636a64945ce47149db6ff002b14402c
I20260619 20:45:13.149169 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:43077
--builtin_ntp_servers=127.13.195.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:45:13.230265 24075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:13.230566 24075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:13.230594 24075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:13.231789 24075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:13.231830 24075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:13.231856 24075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:45:13.233335 24075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:13.233600 24075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:13.234020 24075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:13.239120 24080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:13.239105 24083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:13.239105 24081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:13.239137 24075 server_base.cc:1034] running on GCE node
I20260619 20:45:13.239610 24075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:13.239867 24075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:13.241073 24075 hybrid_clock.cc:648] HybridClock initialized: now 1781901913241046 us; error 38 us; skew 500 ppm
I20260619 20:45:13.242476 24075 webserver.cc:460] Webserver started at http://127.13.195.66:46493/ using document root <none> and password file <none>
I20260619 20:45:13.242759 24075 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:13.242828 24075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:13.242997 24075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:13.244118 24075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "a914b25968804579a5966bd666b22d63"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "5414e48790d47c32ec9003007c4e3fda"
server_key_iv: "85d08ffdca1c80f915e8e82669f10b83"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.244493 24075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "a914b25968804579a5966bd666b22d63"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "5414e48790d47c32ec9003007c4e3fda"
server_key_iv: "85d08ffdca1c80f915e8e82669f10b83"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.245960 24075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:45:13.246810 24089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.247011 24075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:13.247117 24075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "a914b25968804579a5966bd666b22d63"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "5414e48790d47c32ec9003007c4e3fda"
server_key_iv: "85d08ffdca1c80f915e8e82669f10b83"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.247195 24075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:13.250694 24075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:13.250974 24075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:13.251122 24075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:13.251332 24075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:13.251650 24075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:13.251704 24075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.251744 24075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:13.251771 24075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.257617 24075 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:45451
I20260619 20:45:13.257653 24202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:45451 every 8 connection(s)
I20260619 20:45:13.257994 24075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:45:13.261025 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:13.258794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:52055 (local address 127.13.195.126:43077)
0619 20:45:13.258871 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:13.258874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:13.259102 (+ 228us) server_negotiation.cc:407] Connection header received
0619 20:45:13.259596 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:13.259597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:13.259603 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:13.259667 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:13.260817 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:13.260818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:13.260819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:13.260823 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:13.260838 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:13.260842 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:13.260894 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:13.260954 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:45:13.260967 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:45:13.261260 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:13.258751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:43077 (local address 127.13.195.66:52055)
0619 20:45:13.258938 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:13.258973 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:13.259504 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:13.259721 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:13.259724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:13.259732 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:13.260663 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:45:13.260663 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:13.260692 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:45:13.260720 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:13.260893 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:13.260895 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:13.260898 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:45:13.260952 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:45:13.261012 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260619 20:45:13.261773 24203 heartbeater.cc:347] Connected to a master server at 127.13.195.126:43077
I20260619 20:45:13.261866 24203 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:13.262042 24203 heartbeater.cc:511] Master 127.13.195.126:43077 requested a full tablet report, sending...
I20260619 20:45:13.262502 23884 ts_manager.cc:194] Registered new tserver with Master: a914b25968804579a5966bd666b22d63 (127.13.195.66:45451)
I20260619 20:45:13.263841 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 24075
I20260619 20:45:13.263975 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:45:13.264174 14093 external_mini_cluster.cc:1428] Setting key 7e3eceadbafe5618c6ba292a566415f0
I20260619 20:45:13.265128 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:43077
--builtin_ntp_servers=127.13.195.84:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:45:13.345410 24206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:13.345706 24206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:13.345732 24206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:13.346884 24206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:13.346922 24206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:13.346948 24206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:45:13.348248 24206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:13.348429 24206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:13.348675 24206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:13.353786 24214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:13.353808 24212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:13.353782 24211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:13.354038 24206 server_base.cc:1034] running on GCE node
I20260619 20:45:13.354257 24206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:13.354561 24206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:13.355707 24206 hybrid_clock.cc:648] HybridClock initialized: now 1781901913355674 us; error 49 us; skew 500 ppm
I20260619 20:45:13.356930 24206 webserver.cc:460] Webserver started at http://127.13.195.67:35791/ using document root <none> and password file <none>
I20260619 20:45:13.357316 24206 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:13.357375 24206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:13.357515 24206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:13.358707 24206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "3e795a83cb554140a37f66fa8e44b180"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "a72cb5fb06a5142867e049181ff07aac"
server_key_iv: "1e0f901959512d0307357e0c04f9dd03"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.359077 24206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "3e795a83cb554140a37f66fa8e44b180"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "a72cb5fb06a5142867e049181ff07aac"
server_key_iv: "1e0f901959512d0307357e0c04f9dd03"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.360388 24206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:45:13.361140 24220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.361354 24206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:13.361446 24206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "3e795a83cb554140a37f66fa8e44b180"
format_stamp: "Formatted at 2026-06-19 20:45:13 on dist-test-slave-n2td"
server_key: "a72cb5fb06a5142867e049181ff07aac"
server_key_iv: "1e0f901959512d0307357e0c04f9dd03"
server_key_version: "encryptionkey@0"
I20260619 20:45:13.361519 24206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:13.364952 24206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:13.365211 24206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:13.365346 24206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:13.365545 24206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:13.365840 24206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:13.365871 24206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.365929 24206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:13.365957 24206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:13.371562 24206 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:34777
I20260619 20:45:13.371632 24333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:34777 every 8 connection(s)
I20260619 20:45:13.371949 24206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:45:13.375113 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:13.372683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:38789 (local address 127.13.195.126:43077)
0619 20:45:13.372744 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:13.372746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:13.372967 (+ 221us) server_negotiation.cc:407] Connection header received
0619 20:45:13.373499 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:13.373500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:13.373507 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:13.373557 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:13.374818 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:13.374819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:13.374819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:45:13.374824 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:13.374840 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:45:13.374845 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:13.374920 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:13.375026 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 20:45:13.375041 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:45:13.375319 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:13.372675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:43077 (local address 127.13.195.67:38789)
0619 20:45:13.372845 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:13.372877 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:13.373355 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:13.373623 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:13.373627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:13.373636 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:13.374652 (+ 1016us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:45:13.374652 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:13.374679 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:45:13.374705 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:13.374921 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:13.374926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:13.374929 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:45:13.375023 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 20:45:13.375085 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260619 20:45:13.375898 24334 heartbeater.cc:347] Connected to a master server at 127.13.195.126:43077
I20260619 20:45:13.375977 24334 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:13.376134 24334 heartbeater.cc:511] Master 127.13.195.126:43077 requested a full tablet report, sending...
I20260619 20:45:13.376571 23884 ts_manager.cc:194] Registered new tserver with Master: 3e795a83cb554140a37f66fa8e44b180 (127.13.195.67:34777)
I20260619 20:45:13.379923 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 24206
I20260619 20:45:13.380041 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:45:13.380244 14093 external_mini_cluster.cc:1428] Setting key 8d069fd12c8f3e024dca633235da5086
I20260619 20:45:13.381253 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:45:13.383055 23923 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:45:13.382627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.13.195.126:43077)
0619 20:45:13.382681 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:13.382683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:13.382772 (+ 89us) server_negotiation.cc:407] Connection header received
0619 20:45:13.382836 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:13.382836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:13.382840 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:13.382892 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:13.383006 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40130: BlockingRecv error: recv got EOF from 127.0.0.1:40130 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:45:13.383117 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23944
I20260619 20:45:13.388662 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 24075
I20260619 20:45:13.393309 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 24206
I20260619 20:45:13.397565 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 23853
2026-06-19T20:45:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (547 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:45:13 dist-test-slave-n2td krb5kdc[24344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:45:13 dist-test-slave-n2td krb5kdc[24344](info): set up 2 sockets
Jun 19 20:45:13 dist-test-slave-n2td krb5kdc[24344](info): commencing operation
krb5kdc: starting...
W20260619 20:45:14.438139 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:45:14 dist-test-slave-n2td krb5kdc[24344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:45:14.557446 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40355
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:40355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:45:14.639365 24357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:14.639613 24357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:14.639652 24357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:14.640828 24357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:45:14.640904 24357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:14.640929 24357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:14.640959 24357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:45:14.640988 24357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:45:14.642562 24357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:40355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:14.642849 24357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:14.643091 24357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:45:14.648234 24357 server_base.cc:1034] running on GCE node
W20260619 20:45:14.648217 24365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:14.648209 24363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:14.648217 24362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:14.648696 24357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:14.648965 24357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:14.650200 24357 hybrid_clock.cc:648] HybridClock initialized: now 1781901914650183 us; error 34 us; skew 500 ppm
Jun 19 20:45:14 dist-test-slave-n2td krb5kdc[24344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:45:14.652854 24357 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:45:14.653306 24357 webserver.cc:460] Webserver started at http://127.13.195.126:43793/ using document root <none> and password file <none>
I20260619 20:45:14.653563 24357 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:14.653609 24357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:14.653738 24357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:14.654883 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "55d83a8e29da4256a0d8701dbf0cd018"
format_stamp: "Formatted at 2026-06-19 20:45:14 on dist-test-slave-n2td"
server_key: "0b6d09f50102f714d53918417caa3edb"
server_key_iv: "d710ad8e1603e73afde462416be993d2"
server_key_version: "encryptionkey@0"
I20260619 20:45:14.655230 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "55d83a8e29da4256a0d8701dbf0cd018"
format_stamp: "Formatted at 2026-06-19 20:45:14 on dist-test-slave-n2td"
server_key: "0b6d09f50102f714d53918417caa3edb"
server_key_iv: "d710ad8e1603e73afde462416be993d2"
server_key_version: "encryptionkey@0"
I20260619 20:45:14.656932 24357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:45:14.658026 24372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:14.658193 24357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:14.658282 24357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "55d83a8e29da4256a0d8701dbf0cd018"
format_stamp: "Formatted at 2026-06-19 20:45:14 on dist-test-slave-n2td"
server_key: "0b6d09f50102f714d53918417caa3edb"
server_key_iv: "d710ad8e1603e73afde462416be993d2"
server_key_version: "encryptionkey@0"
I20260619 20:45:14.658355 24357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:14.675339 24357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:14.677711 24357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:14.677874 24357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:14.681813 24357 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:40355
I20260619 20:45:14.681885 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:40355 every 8 connection(s)
I20260619 20:45:14.682243 24357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:45:14.682559 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 24357
I20260619 20:45:14.682663 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:45:14.682906 14093 external_mini_cluster.cc:1428] Setting key 214723df2b28dd3eff13326b568014f1
I20260619 20:45:14.683543 24425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:45:14.686703 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018: Bootstrap starting.
I20260619 20:45:14.687377 24425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018: Neither blocks nor log segments found. Creating new log.
I20260619 20:45:14.687675 24425 log.cc:824] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018: Log is configured to *not* fsync() on all Append() calls
I20260619 20:45:14.688421 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018: No bootstrap required, opened a new log
Jun 19 20:45:14 dist-test-slave-n2td krb5kdc[24344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:45:14.690663 24425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d83a8e29da4256a0d8701dbf0cd018" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40355 } }
I20260619 20:45:14.690826 24425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:45:14.690851 24425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d83a8e29da4256a0d8701dbf0cd018, State: Initialized, Role: FOLLOWER
I20260619 20:45:14.691002 24425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d83a8e29da4256a0d8701dbf0cd018" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40355 } }
I20260619 20:45:14.691110 24425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:45:14.691155 24425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:45:14.691215 24425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:45:14.691977 24425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d83a8e29da4256a0d8701dbf0cd018" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40355 } }
I20260619 20:45:14.692104 24425 leader_election.cc:302] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55d83a8e29da4256a0d8701dbf0cd018; no voters:
I20260619 20:45:14.692267 24425 leader_election.cc:288] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:45:14.692351 24430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:45:14.692518 24425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:45:14.692575 24430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [term 1 LEADER]: Becoming Leader. State: Replica: 55d83a8e29da4256a0d8701dbf0cd018, State: Running, Role: LEADER
I20260619 20:45:14.692680 24430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d83a8e29da4256a0d8701dbf0cd018" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40355 } }
I20260619 20:45:14.693023 24430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55d83a8e29da4256a0d8701dbf0cd018. Latest consensus state: current_term: 1 leader_uuid: "55d83a8e29da4256a0d8701dbf0cd018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d83a8e29da4256a0d8701dbf0cd018" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40355 } } }
I20260619 20:45:14.693112 24430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [sys.catalog]: This master's current role is: LEADER
I20260619 20:45:14.693035 24432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55d83a8e29da4256a0d8701dbf0cd018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d83a8e29da4256a0d8701dbf0cd018" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40355 } } }
I20260619 20:45:14.693252 24432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018 [sys.catalog]: This master's current role is: LEADER
W20260619 20:45:14.694470 24445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:45:14.694573 24445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:45:14.694689 24438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:45:14.694876 24438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:45:14.694993 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:14.683466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.13.195.126:40355)
0619 20:45:14.683701 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:14.683710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:14.683740 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:45:14.684292 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:14.684301 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:14.684364 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:14.684525 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:14.685089 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:14.686459 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:14.687263 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:14.687906 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:14.688000 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:14.690707 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:14.690711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:14.690711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:45:14.690722 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:14.692786 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:14.693239 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:14.693243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:14.693244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:14.693389 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:14.693639 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:14.693640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:14.693640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:14.693985 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:14.694559 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:14.694630 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 20:45:14.694767 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
I20260619 20:45:14.696441 24438 catalog_manager.cc:1347] Generated new cluster ID: 416841b89e224f889d30bcd4d9c1558b
I20260619 20:45:14.696506 24438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:45:14.703474 24438 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:45:14.704042 24438 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:45:14.714735 24438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018: Generated new TSK 0
I20260619 20:45:14.714953 24438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:45:14.751403 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40355
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:45:14.834651 24451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:14.834867 24451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:14.834892 24451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:14.836105 24451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:14.836181 24451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:14.836230 24451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:45:14.837746 24451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:14.838029 24451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:14.838272 24451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:14.843387 24459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:14.843372 24456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:14.843369 24457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:14.843732 24451 server_base.cc:1034] running on GCE node
I20260619 20:45:14.843945 24451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:14.844200 24451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:14.845386 24451 hybrid_clock.cc:648] HybridClock initialized: now 1781901914845372 us; error 43 us; skew 500 ppm
Jun 19 20:45:14 dist-test-slave-n2td krb5kdc[24344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:45:14.848053 24451 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:45:14.848467 24451 webserver.cc:460] Webserver started at http://127.13.195.65:40287/ using document root <none> and password file <none>
I20260619 20:45:14.848719 24451 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:14.848765 24451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:14.848897 24451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:14.849967 24451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "abccdeb7fb0d452e91d2929048c0624d"
format_stamp: "Formatted at 2026-06-19 20:45:14 on dist-test-slave-n2td"
server_key: "f41bc795b733ad3e35d1d686e28aa327"
server_key_iv: "f874226688c07c7aa42dbcb755eab44b"
server_key_version: "encryptionkey@0"
I20260619 20:45:14.850286 24451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "abccdeb7fb0d452e91d2929048c0624d"
format_stamp: "Formatted at 2026-06-19 20:45:14 on dist-test-slave-n2td"
server_key: "f41bc795b733ad3e35d1d686e28aa327"
server_key_iv: "f874226688c07c7aa42dbcb755eab44b"
server_key_version: "encryptionkey@0"
I20260619 20:45:14.851729 24451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:45:14.852440 24466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:14.852632 24451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:14.852712 24451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "abccdeb7fb0d452e91d2929048c0624d"
format_stamp: "Formatted at 2026-06-19 20:45:14 on dist-test-slave-n2td"
server_key: "f41bc795b733ad3e35d1d686e28aa327"
server_key_iv: "f874226688c07c7aa42dbcb755eab44b"
server_key_version: "encryptionkey@0"
I20260619 20:45:14.852766 24451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:14.859615 24451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:14.861907 24451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:14.862028 24451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:14.862206 24451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:14.862520 24451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:14.862558 24451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:14.862579 24451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:14.862627 24451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:14.868168 24451 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:45205
I20260619 20:45:14.868229 24579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:45205 every 8 connection(s)
I20260619 20:45:14.868583 24451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:45:14 dist-test-slave-n2td krb5kdc[24344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:45:14.876827 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 24451
I20260619 20:45:14.876950 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:45:14.877153 14093 external_mini_cluster.cc:1428] Setting key de31edbf9d1987141ffbfcacc8a0890d
I20260619 20:45:14.879155 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:14.869372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:56335 (local address 127.13.195.126:40355)
0619 20:45:14.869432 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:14.869434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:14.869711 (+ 277us) server_negotiation.cc:407] Connection header received
0619 20:45:14.870296 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:14.870298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:14.870311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:14.870368 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:14.871428 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:14.872025 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:14.873202 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:14.873446 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:14.873536 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:14.875964 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:14.875965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:14.875966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:45:14.875975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:14.877537 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:14.878053 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:14.878054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:14.878055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:14.878124 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:14.878386 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:14.878387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:14.878388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:14.878582 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:14.878668 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:14.878965 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 20:45:14.879063 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:45:14.879369 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:14.869344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40355 (local address 127.13.195.65:56335)
0619 20:45:14.869566 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:14.869605 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:14.870171 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:14.870471 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:14.870476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:14.871102 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:14.871309 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:14.871313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:14.872169 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:14.872170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:14.873097 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:14.873099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:14.873550 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:14.873551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:14.873686 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:14.874190 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:14.874199 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:14.875848 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:14.877630 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:45:14.877636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:45:14.877638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:45:14.877963 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:45:14.878193 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:45:14.878194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:45:14.878194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:45:14.878311 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:45:14.878656 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:14.878663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:14.878815 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 20:45:14.878944 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 20:45:14.879109 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":89,"threads_started":1}
I20260619 20:45:14.879806 24580 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40355
I20260619 20:45:14.879930 24580 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:14.880119 24580 heartbeater.cc:511] Master 127.13.195.126:40355 requested a full tablet report, sending...
I20260619 20:45:14.880663 24389 ts_manager.cc:194] Registered new tserver with Master: abccdeb7fb0d452e91d2929048c0624d (127.13.195.65:45205)
I20260619 20:45:14.881280 24389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:56335
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:45:14.906579 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40355
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:45:14.988608 24585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:14.988813 24585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:14.988837 24585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:14.989924 24585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:14.989956 24585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:14.989981 24585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:45:14.991423 24585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:14.991612 24585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:14.991784 24585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:14.997000 24593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:14.997102 24585 server_base.cc:1034] running on GCE node
W20260619 20:45:14.996986 24590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:14.996987 24591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:14.997447 24585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:14.997697 24585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:14.998908 24585 hybrid_clock.cc:648] HybridClock initialized: now 1781901914998908 us; error 40 us; skew 500 ppm
Jun 19 20:45:14 dist-test-slave-n2td krb5kdc[24344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:45:15.001439 24585 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:45:15.001864 24585 webserver.cc:460] Webserver started at http://127.13.195.66:32887/ using document root <none> and password file <none>
I20260619 20:45:15.002094 24585 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:15.002161 24585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:15.002314 24585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:15.003432 24585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b"
format_stamp: "Formatted at 2026-06-19 20:45:15 on dist-test-slave-n2td"
server_key: "db776ddf387aacc789c38ce9e2f4e1f8"
server_key_iv: "c9c156443d58bfa03a927479643b759c"
server_key_version: "encryptionkey@0"
I20260619 20:45:15.003801 24585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b"
format_stamp: "Formatted at 2026-06-19 20:45:15 on dist-test-slave-n2td"
server_key: "db776ddf387aacc789c38ce9e2f4e1f8"
server_key_iv: "c9c156443d58bfa03a927479643b759c"
server_key_version: "encryptionkey@0"
I20260619 20:45:15.005154 24585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:45:15.005913 24600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:15.006098 24585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:15.006193 24585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b"
format_stamp: "Formatted at 2026-06-19 20:45:15 on dist-test-slave-n2td"
server_key: "db776ddf387aacc789c38ce9e2f4e1f8"
server_key_iv: "c9c156443d58bfa03a927479643b759c"
server_key_version: "encryptionkey@0"
I20260619 20:45:15.006278 24585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:15.016039 24585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:15.018121 24585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:15.018307 24585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:15.018586 24585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:15.018905 24585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:15.018967 24585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:15.019011 24585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:15.019040 24585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:15.024926 24585 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:33737
I20260619 20:45:15.024977 24713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:33737 every 8 connection(s)
I20260619 20:45:15.025326 24585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:45:15 dist-test-slave-n2td krb5kdc[24344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901914, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:45:15.031610 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 24585
I20260619 20:45:15.031714 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:45:15.031922 14093 external_mini_cluster.cc:1428] Setting key f15d47f5125086eda3e9a6c3c8decbd2
I20260619 20:45:15.034845 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.026154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:37185 (local address 127.13.195.126:40355)
0619 20:45:15.026238 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.026240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.026375 (+ 135us) server_negotiation.cc:407] Connection header received
0619 20:45:15.026975 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.026977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.026987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.027049 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:15.027965 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.028472 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.029441 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.029629 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.029682 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.031790 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:15.031792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:15.031792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:45:15.031800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:15.033358 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:15.033874 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:15.033875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:15.033875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:15.033933 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:15.034200 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:15.034200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:15.034201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:15.034299 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:15.034352 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.034703 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.034779 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:45:15.035216 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.026087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40355 (local address 127.13.195.66:37185)
0619 20:45:15.026238 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.026284 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:15.026850 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:15.027104 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:15.027107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:15.027675 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:15.027876 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.027881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.028571 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.028572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.029365 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.029367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.029701 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.029702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.029798 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.030282 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:15.030290 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:15.031661 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:15.033468 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:45:15.033475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:45:15.033478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:45:15.033781 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:45:15.033996 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:45:15.033997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:45:15.033998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:45:15.034135 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:45:15.034361 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:15.034367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:15.034559 (+ 192us) client_negotiation.cc:769] Sending connection context
0619 20:45:15.034673 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 20:45:15.034854 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":94,"threads_started":1}
I20260619 20:45:15.035732 24714 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40355
I20260619 20:45:15.035827 24714 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:15.036026 24714 heartbeater.cc:511] Master 127.13.195.126:40355 requested a full tablet report, sending...
I20260619 20:45:15.036419 24389 ts_manager.cc:194] Registered new tserver with Master: d0cef3a9ad9b41bdb1d77edc9c7e0c7b (127.13.195.66:33737)
I20260619 20:45:15.036811 24389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:37185
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:45:15.058939 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40355
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:45:15.139475 24719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:15.139700 24719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:15.139740 24719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:15.140903 24719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:15.140971 24719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:15.141017 24719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:45:15.142546 24719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:15.142795 24719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:15.143029 24719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:15.147859 24724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:15.147954 24725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:15.147856 24727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:15.148192 24719 server_base.cc:1034] running on GCE node
I20260619 20:45:15.148386 24719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:15.148665 24719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:15.149823 24719 hybrid_clock.cc:648] HybridClock initialized: now 1781901915149823 us; error 44 us; skew 500 ppm
Jun 19 20:45:15 dist-test-slave-n2td krb5kdc[24344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901915, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:45:15.152498 24719 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:45:15.152935 24719 webserver.cc:460] Webserver started at http://127.13.195.67:34159/ using document root <none> and password file <none>
I20260619 20:45:15.153182 24719 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:15.153230 24719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:15.153362 24719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:45:15.154481 24719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "855306a373eb4e17a237c41e5963d900"
format_stamp: "Formatted at 2026-06-19 20:45:15 on dist-test-slave-n2td"
server_key: "29af4ba80090a80baadcf08e0ad7c236"
server_key_iv: "1b06a45d6a0d815f6b92e7f2b43e2e04"
server_key_version: "encryptionkey@0"
I20260619 20:45:15.154857 24719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "855306a373eb4e17a237c41e5963d900"
format_stamp: "Formatted at 2026-06-19 20:45:15 on dist-test-slave-n2td"
server_key: "29af4ba80090a80baadcf08e0ad7c236"
server_key_iv: "1b06a45d6a0d815f6b92e7f2b43e2e04"
server_key_version: "encryptionkey@0"
I20260619 20:45:15.156188 24719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:15.156937 24734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:15.157152 24719 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:45:15.157254 24719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "855306a373eb4e17a237c41e5963d900"
format_stamp: "Formatted at 2026-06-19 20:45:15 on dist-test-slave-n2td"
server_key: "29af4ba80090a80baadcf08e0ad7c236"
server_key_iv: "1b06a45d6a0d815f6b92e7f2b43e2e04"
server_key_version: "encryptionkey@0"
I20260619 20:45:15.157336 24719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:15.168591 24719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:15.170588 24719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:15.170759 24719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:15.171016 24719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:45:15.171352 24719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:45:15.171388 24719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:15.171438 24719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:45:15.171476 24719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:15.177768 24719 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:40143
I20260619 20:45:15.177825 24847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:40143 every 8 connection(s)
I20260619 20:45:15.178164 24719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:45:15.183590 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 24719
I20260619 20:45:15.183727 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:45:15.183959 14093 external_mini_cluster.cc:1428] Setting key 038561822aba822180f6daa420fde81c
Jun 19 20:45:15 dist-test-slave-n2td krb5kdc[24344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901915, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:45:15.188046 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.178930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:45987 (local address 127.13.195.126:40355)
0619 20:45:15.179027 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.179029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.179274 (+ 245us) server_negotiation.cc:407] Connection header received
0619 20:45:15.179811 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.179812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.179823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.179883 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:15.180875 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.181388 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.182397 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.182713 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.182778 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.185281 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:15.185283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:15.185284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:45:15.185292 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:15.186622 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:15.187045 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:15.187046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:15.187046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:15.187104 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:15.187368 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:15.187368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:15.187369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:15.187520 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:15.187601 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.187874 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.187951 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 20:45:15.188273 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.178919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40355 (local address 127.13.195.67:45987)
0619 20:45:15.179144 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.179182 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:15.179713 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:15.179939 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:15.179942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:15.180560 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:45:15.180769 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.180774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.181504 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.181506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.182303 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.182306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.182792 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.182792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.182896 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.183479 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:45:15.183487 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:45:15.185179 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:45:15.186715 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:45:15.186722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:45:15.186724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:45:15.186954 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:45:15.187186 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:45:15.187187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:45:15.187187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:45:15.187295 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:45:15.187592 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:45:15.187598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:45:15.187739 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 20:45:15.187856 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 20:45:15.188021 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":112,"threads_started":1}
I20260619 20:45:15.188657 24848 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40355
I20260619 20:45:15.188763 24848 heartbeater.cc:464] Registering TS with master...
I20260619 20:45:15.188949 24848 heartbeater.cc:511] Master 127.13.195.126:40355 requested a full tablet report, sending...
I20260619 20:45:15.189273 24389 ts_manager.cc:194] Registered new tserver with Master: 855306a373eb4e17a237c41e5963d900 (127.13.195.67:40143)
I20260619 20:45:15.189678 24389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:45987
I20260619 20:45:15.195688 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:45:15.202807 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.196884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.13.195.126:40355)
0619 20:45:15.196953 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.196955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.197030 (+ 75us) server_negotiation.cc:407] Connection header received
0619 20:45:15.197085 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.197086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.197095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.197135 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:15.197638 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.198111 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.198877 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.199085 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.199166 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.200302 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:15.200304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:15.200304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:45:15.200311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:15.201652 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:15.201976 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:15.201976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:15.201977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:15.202030 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:15.202192 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:15.202192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:15.202192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:15.202331 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:15.202525 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.202602 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.202663 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:45:15.204548 24389 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:45:15.206269 24389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:45:15.213171 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.209006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:45205 (local address 127.0.0.1:52392)
0619 20:45:15.209206 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.209213 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:15.209238 (+ 25us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:15.209633 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:15.209634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:15.209637 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:45:15.209692 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.209693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.211084 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.211085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.212099 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.212102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.212905 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.212906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.212986 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.212999 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:45:15.213067 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:45:15.213108 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":47,"threads_started":1}
I20260619 20:45:15.213232 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.209223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.13.195.65:45205)
0619 20:45:15.209472 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.209475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.209484 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:45:15.209521 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.209523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.209537 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.209765 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:45:15.209840 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.210955 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.212182 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.212811 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.212887 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.212944 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.213100 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.213143 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
I20260619 20:45:15.213989 24514 tablet_service.cc:1467] Processing CreateTablet for tablet 122e7ab053074e67a6745ae0825927be (DEFAULT_TABLE table=test-table [id=b1affdb7e3424785ae92454cb3bc6e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:45:15.214392 24514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122e7ab053074e67a6745ae0825927be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:45:15.214473 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.208770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:40143 (local address 127.0.0.1:58438)
0619 20:45:15.208927 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.208971 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:15.209039 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:15.209562 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:15.209563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:15.209565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:45:15.209632 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.209635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.210943 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.210945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.212416 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.212417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.214174 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.214175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.214242 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.214246 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:45:15.214304 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:45:15.214355 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":8,"thread_start_us":61,"threads_started":1}
I20260619 20:45:15.214568 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.208867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58438 (local address 127.13.195.67:40143)
0619 20:45:15.209358 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.209362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.209372 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:45:15.209414 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.209416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.209441 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.209558 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:45:15.209729 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.210810 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.213417 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.214101 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.214165 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.214236 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.214421 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.214489 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":113,"threads_started":1}
I20260619 20:45:15.214589 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.208770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:33737 (local address 127.0.0.1:45546)
0619 20:45:15.209088 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.209097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:15.209140 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:15.209528 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:15.209531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:15.209534 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:45:15.209679 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.209681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.211066 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.211067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.212407 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.212411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.214317 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.214318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.214386 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.214392 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:45:15.214475 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:45:15.214517 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":127,"thread_start_us":81,"threads_started":1}
I20260619 20:45:15.214844 24860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.209013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45546 (local address 127.13.195.66:33737)
0619 20:45:15.209231 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.209237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.209250 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:45:15.209298 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.209301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.209320 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.209419 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:45:15.209769 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.210955 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.212507 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.213259 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.214305 (+ 1046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.214366 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.214687 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.214745 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":91,"threads_started":1}
I20260619 20:45:15.215296 24782 tablet_service.cc:1467] Processing CreateTablet for tablet 122e7ab053074e67a6745ae0825927be (DEFAULT_TABLE table=test-table [id=b1affdb7e3424785ae92454cb3bc6e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:45:15.215471 24648 tablet_service.cc:1467] Processing CreateTablet for tablet 122e7ab053074e67a6745ae0825927be (DEFAULT_TABLE table=test-table [id=b1affdb7e3424785ae92454cb3bc6e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:45:15.215634 24782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122e7ab053074e67a6745ae0825927be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:45:15.215799 24648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122e7ab053074e67a6745ae0825927be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:45:15.217963 24862 tablet_bootstrap.cc:492] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d: Bootstrap starting.
I20260619 20:45:15.218577 24863 tablet_bootstrap.cc:492] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Bootstrap starting.
I20260619 20:45:15.218763 24862 tablet_bootstrap.cc:654] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d: Neither blocks nor log segments found. Creating new log.
I20260619 20:45:15.219105 24864 tablet_bootstrap.cc:492] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Bootstrap starting.
I20260619 20:45:15.219238 24863 tablet_bootstrap.cc:654] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Neither blocks nor log segments found. Creating new log.
I20260619 20:45:15.219287 24862 log.cc:824] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:45:15.219772 24864 tablet_bootstrap.cc:654] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Neither blocks nor log segments found. Creating new log.
I20260619 20:45:15.219834 24863 log.cc:824] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:45:15.220084 24862 tablet_bootstrap.cc:492] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d: No bootstrap required, opened a new log
I20260619 20:45:15.220155 24862 ts_tablet_manager.cc:1397] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:45:15.220371 24864 log.cc:824] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Log is configured to *not* fsync() on all Append() calls
I20260619 20:45:15.221045 24864 tablet_bootstrap.cc:492] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: No bootstrap required, opened a new log
I20260619 20:45:15.221097 24864 ts_tablet_manager.cc:1397] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:45:15.221740 24862 raft_consensus.cc:348] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.221894 24862 raft_consensus.cc:374] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:45:15.221938 24862 raft_consensus.cc:729] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abccdeb7fb0d452e91d2929048c0624d, State: Initialized, Role: FOLLOWER
I20260619 20:45:15.222077 24862 consensus_queue.cc:260] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.222368 24862 ts_tablet_manager.cc:1428] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:45:15.222404 24580 heartbeater.cc:503] Master 127.13.195.126:40355 was elected leader, sending a full tablet report...
I20260619 20:45:15.222632 24864 raft_consensus.cc:348] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.222777 24864 raft_consensus.cc:374] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:45:15.222810 24864 raft_consensus.cc:729] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855306a373eb4e17a237c41e5963d900, State: Initialized, Role: FOLLOWER
I20260619 20:45:15.222919 24864 consensus_queue.cc:260] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.223188 24864 ts_tablet_manager.cc:1428] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:45:15.223209 24848 heartbeater.cc:503] Master 127.13.195.126:40355 was elected leader, sending a full tablet report...
I20260619 20:45:15.223649 24863 tablet_bootstrap.cc:492] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: No bootstrap required, opened a new log
I20260619 20:45:15.223748 24863 ts_tablet_manager.cc:1397] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 20:45:15.225378 24863 raft_consensus.cc:348] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.225534 24863 raft_consensus.cc:374] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:45:15.225581 24863 raft_consensus.cc:729] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0cef3a9ad9b41bdb1d77edc9c7e0c7b, State: Initialized, Role: FOLLOWER
I20260619 20:45:15.225687 24863 consensus_queue.cc:260] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.225941 24863 ts_tablet_manager.cc:1428] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:45:15.226001 24714 heartbeater.cc:503] Master 127.13.195.126:40355 was elected leader, sending a full tablet report...
W20260619 20:45:15.275920 24715 tablet.cc:2369] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:45:15.369395 24581 tablet.cc:2369] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:45:15.428779 24849 tablet.cc:2369] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:45:15.481076 24869 raft_consensus.cc:482] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:45:15.481207 24869 raft_consensus.cc:504] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.481570 24869 leader_election.cc:288] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0cef3a9ad9b41bdb1d77edc9c7e0c7b (127.13.195.66:33737), abccdeb7fb0d452e91d2929048c0624d (127.13.195.65:45205)
I20260619 20:45:15.484629 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.481935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:49931 (local address 127.13.195.65:45205)
0619 20:45:15.481980 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.481983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.481991 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:45:15.482024 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.482025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.482031 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.482091 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:45:15.482252 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.482951 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.483902 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.484264 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.484357 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.484367 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.484529 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.484566 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:45:15.484629 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.481790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:45205 (local address 127.13.195.67:49931)
0619 20:45:15.481850 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.481865 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:15.481916 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:15.482109 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:15.482110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:15.482111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:45:15.482181 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.482183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.483053 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.483054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.483797 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.483799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.484357 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.484358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.484443 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.484446 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:45:15.484508 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:45:15.484551 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:45:15.484764 24860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.481912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:54095 (local address 127.13.195.66:33737)
0619 20:45:15.481985 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.481988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.482066 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:45:15.482128 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.482129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.482137 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.482208 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:45:15.482485 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.483089 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.484030 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.484408 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.484504 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.484513 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.484667 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.484702 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:45:15.484773 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.481801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:33737 (local address 127.13.195.67:54095)
0619 20:45:15.481982 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.481993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:45:15.482061 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:45:15.482212 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:45:15.482213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:45:15.482214 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:45:15.482289 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.482290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.483185 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.483186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.483933 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:45:15.483934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.484505 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.484506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:45:15.484584 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.484587 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:45:15.484649 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:45:15.484691 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":21,"thread_start_us":90,"threads_started":1}
I20260619 20:45:15.485164 24534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122e7ab053074e67a6745ae0825927be" candidate_uuid: "855306a373eb4e17a237c41e5963d900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abccdeb7fb0d452e91d2929048c0624d" is_pre_election: true
I20260619 20:45:15.485159 24668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122e7ab053074e67a6745ae0825927be" candidate_uuid: "855306a373eb4e17a237c41e5963d900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" is_pre_election: true
I20260619 20:45:15.485332 24668 raft_consensus.cc:2454] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 855306a373eb4e17a237c41e5963d900 in term 0.
I20260619 20:45:15.485332 24534 raft_consensus.cc:2454] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 855306a373eb4e17a237c41e5963d900 in term 0.
I20260619 20:45:15.485598 24738 leader_election.cc:302] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 855306a373eb4e17a237c41e5963d900, abccdeb7fb0d452e91d2929048c0624d; no voters:
I20260619 20:45:15.485756 24869 raft_consensus.cc:2781] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:45:15.485812 24869 raft_consensus.cc:482] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:45:15.485841 24869 raft_consensus.cc:3037] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:45:15.486624 24869 raft_consensus.cc:504] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.486769 24869 leader_election.cc:288] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [CANDIDATE]: Term 1 election: Requested vote from peers d0cef3a9ad9b41bdb1d77edc9c7e0c7b (127.13.195.66:33737), abccdeb7fb0d452e91d2929048c0624d (127.13.195.65:45205)
I20260619 20:45:15.486959 24668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122e7ab053074e67a6745ae0825927be" candidate_uuid: "855306a373eb4e17a237c41e5963d900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b"
I20260619 20:45:15.486948 24534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122e7ab053074e67a6745ae0825927be" candidate_uuid: "855306a373eb4e17a237c41e5963d900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abccdeb7fb0d452e91d2929048c0624d"
I20260619 20:45:15.487049 24668 raft_consensus.cc:3037] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:45:15.487047 24534 raft_consensus.cc:3037] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:45:15.487740 24668 raft_consensus.cc:2454] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 855306a373eb4e17a237c41e5963d900 in term 1.
I20260619 20:45:15.487869 24534 raft_consensus.cc:2454] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 855306a373eb4e17a237c41e5963d900 in term 1.
I20260619 20:45:15.487969 24735 leader_election.cc:302] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 855306a373eb4e17a237c41e5963d900, d0cef3a9ad9b41bdb1d77edc9c7e0c7b; no voters:
I20260619 20:45:15.488058 24869 raft_consensus.cc:2781] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:45:15.488248 24869 raft_consensus.cc:686] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 1 LEADER]: Becoming Leader. State: Replica: 855306a373eb4e17a237c41e5963d900, State: Running, Role: LEADER
I20260619 20:45:15.488353 24869 consensus_queue.cc:237] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } }
I20260619 20:45:15.489231 24388 catalog_manager.cc:5495] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 855306a373eb4e17a237c41e5963d900 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "855306a373eb4e17a237c41e5963d900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "855306a373eb4e17a237c41e5963d900" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 40143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 } health_report { overall_health: UNKNOWN } } }
I20260619 20:45:15.494789 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.492570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.13.195.67:40143)
0619 20:45:15.492613 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:15.492618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:15.492633 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:45:15.492678 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:15.492679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:15.492693 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:15.492748 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:45:15.492852 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.493302 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.494071 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:15.494246 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:15.494299 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:15.494518 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:45:15.494613 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:45:15.494678 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:15.494709 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 20:45:15.494725 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:45:15.496475 24534 raft_consensus.cc:1261] T 122e7ab053074e67a6745ae0825927be P abccdeb7fb0d452e91d2929048c0624d [term 1 FOLLOWER]: Refusing update from remote peer 855306a373eb4e17a237c41e5963d900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:45:15.496475 24668 raft_consensus.cc:1261] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 1 FOLLOWER]: Refusing update from remote peer 855306a373eb4e17a237c41e5963d900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:45:15.496806 24869 consensus_queue.cc:1035] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abccdeb7fb0d452e91d2929048c0624d" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 45205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:45:15.496938 24872 consensus_queue.cc:1035] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0cef3a9ad9b41bdb1d77edc9c7e0c7b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:45:15.500339 24876 mvcc.cc:205] Tried to move back new op lower bound from 7298670245870907392 to 7298670245841174528. Current Snapshot: MvccSnapshot[applied={T|T < 7298670245870907392}]
I20260619 20:45:15.501353 24389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:45:15.501446 24389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:45:15.502154 24389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55d83a8e29da4256a0d8701dbf0cd018: Sending DeleteTablet for 3 replicas of tablet 122e7ab053074e67a6745ae0825927be
I20260619 20:45:15.502521 24782 tablet_service.cc:1514] Processing DeleteTablet for tablet 122e7ab053074e67a6745ae0825927be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:45:15 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:58438
I20260619 20:45:15.502554 24648 tablet_service.cc:1514] Processing DeleteTablet for tablet 122e7ab053074e67a6745ae0825927be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:45:15 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:45546
I20260619 20:45:15.502544 24514 tablet_service.cc:1514] Processing DeleteTablet for tablet 122e7ab053074e67a6745ae0825927be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:45:15 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:52392
I20260619 20:45:15.503062 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 24451
I20260619 20:45:15.506516 24885 tablet_replica.cc:330] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: stopping tablet replica
I20260619 20:45:15.506585 24886 tablet_replica.cc:330] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: stopping tablet replica
I20260619 20:45:15.506630 24885 raft_consensus.cc:2229] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:45:15.506731 24885 raft_consensus.cc:2258] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:45:15.506719 24886 raft_consensus.cc:2229] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:45:15.506853 24886 raft_consensus.cc:2258] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:45:15.507398 24885 ts_tablet_manager.cc:1905] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:45:15.507462 24886 ts_tablet_manager.cc:1905] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:45:15.509269 24885 ts_tablet_manager.cc:1918] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:45:15.509269 24886 ts_tablet_manager.cc:1918] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:45:15.509349 24886 log.cc:1196] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/122e7ab053074e67a6745ae0825927be
I20260619 20:45:15.509459 24885 log.cc:1196] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/122e7ab053074e67a6745ae0825927be
I20260619 20:45:15.509696 24886 ts_tablet_manager.cc:1939] T 122e7ab053074e67a6745ae0825927be P d0cef3a9ad9b41bdb1d77edc9c7e0c7b: Deleting consensus metadata
I20260619 20:45:15.509917 24885 ts_tablet_manager.cc:1939] T 122e7ab053074e67a6745ae0825927be P 855306a373eb4e17a237c41e5963d900: Deleting consensus metadata
I20260619 20:45:15.510223 24373 catalog_manager.cc:4841] TS d0cef3a9ad9b41bdb1d77edc9c7e0c7b (127.13.195.66:33737): tablet 122e7ab053074e67a6745ae0825927be (table test-table [id=b1affdb7e3424785ae92454cb3bc6e8a]) successfully deleted
I20260619 20:45:15.510414 24376 catalog_manager.cc:4841] TS 855306a373eb4e17a237c41e5963d900 (127.13.195.67:40143): tablet 122e7ab053074e67a6745ae0825927be (table test-table [id=b1affdb7e3424785ae92454cb3bc6e8a]) successfully deleted
W20260619 20:45:15.511194 24376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:45205 (error 108)
I20260619 20:45:15.511404 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 24585
I20260619 20:45:15.511998 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:15.511599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:45205 (local address 127.0.0.1:44110)
0619 20:45:15.511909 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 20:45:15.511959 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:45205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286}
W20260619 20:45:15.513254 24376 catalog_manager.cc:4584] TS abccdeb7fb0d452e91d2929048c0624d (127.13.195.65:45205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 122e7ab053074e67a6745ae0825927be: Network error: Client connection negotiation failed: client connection to 127.13.195.65:45205: connect: Connection refused (error 111)
I20260619 20:45:15.517007 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 24719
I20260619 20:45:15.522197 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 24357
2026-06-19T20:45:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2123 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:45:15 dist-test-slave-n2td krb5kdc[24891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:45:15 dist-test-slave-n2td krb5kdc[24891](info): set up 2 sockets
Jun 19 20:45:15 dist-test-slave-n2td krb5kdc[24891](info): commencing operation
krb5kdc: starting...
W20260619 20:45:16.610368 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:45:16 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:45:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:45:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/rangeradmin_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/rangeradmin_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/rangerlookup_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/rangerlookup_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
I20260619 20:45:16.776804 14093 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskhOhR3l/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/postgres -l logfile start
2026-06-19 20:45:19.228 UTC [24921] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 20:45:19.228 UTC [24921] LOG: listening on IPv4 address "127.13.195.84", port 46065
2026-06-19 20:45:19.232 UTC [24921] LOG: listening on Unix socket "/tmp/.s.PGSQL.46065"
2026-06-19 20:45:19.254 UTC [24924] LOG: database system was shut down at 2026-06-19 20:45:17 UTC
2026-06-19 20:45:19.258 UTC [24921] LOG: database system is ready to accept connections
I20260619 20:45:20.238862 14093 mini_postgres.cc:96] Postgres bound to 46065
2026-06-19 20:45:20.245 UTC [24932] FATAL: database "slave" does not exist
127.13.195.84:46065 - accepting connections
I20260619 20:45:20.245865 14093 mini_ranger.cc:161] Starting Ranger...
I20260619 20:45:20.261545 14093 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 20:45:20.736305 14093 mini_ranger.cc:87] Created ranger Postgres database
I20260619 20:45:20.736377 14093 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-admin
2026-06-19 20:45:21,029 [I] DB FLAVOR :POSTGRES
2026-06-19 20:45:21,030 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:45:21,030 [I] Checking connection..
2026-06-19 20:45:21,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 20:45:21,302 [I] Checking connection passed.
2026-06-19 20:45:21,302 [I] --------- Verifying version history table ---------
2026-06-19 20:45:21,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:45:21,575 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:45:21,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:45:21,834 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:45:21,834 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 20:45:21,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 20:45:22.059 UTC [24996] WARNING: there is no transaction in progress
2026-06-19 20:45:22,066 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 20:45:22,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:45:22,333 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 20:45:22,333 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:45:22,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 20:45:22,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 20:45:22,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-n2td.c.gcp-upstream.internal','N') ;"
2026-06-19 20:45:23,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:45:23,329 [I] Table x_portal_user does not exist in database ranger
2026-06-19 20:45:23,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:45:23,583 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 20:45:23,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:45:23,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:45:24,100 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 20:45:24,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 20:45:25.062 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.078 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.096 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.111 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.127 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.148 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.203 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.210 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.225 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.237 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.248 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.263 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.274 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.281 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.287 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25.294 UTC [25104] WARNING: there is no transaction in progress
2026-06-19 20:45:25,979 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 20:45:25,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 20:45:26,227 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 20:45:26,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:45:26,472 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 20:45:26,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:45:26,722 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 20:45:26,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:45:26,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:45:27,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-n2td.c.gcp-upstream.internal';"
2026-06-19 20:45:27,484 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 20:45:27,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 20:45:27,759 [I] DB_PATCHES have already been applied
I20260619 20:45:27.765146 14093 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-admin:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskhOhR3l/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 20:45:27.765213 14093 mini_ranger.cc:193] Using host: 127.13.195.84
I20260619 20:45:27.768174 14093 mini_ranger.cc:227] Ranger admin URL: http://127.13.195.84:57695
Jun 19, 2026 8:45:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:45:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:45:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:45:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:45:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:45:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:45:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 20:45:28.970856 14093 mini_ranger.cc:160] Time spent starting Ranger: real 8.725s user 0.000s sys 0.009s
Jun 19, 2026 8:45:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:45:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 8:45:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:45:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:45:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 20:45:51.195705 14093 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/rangerkms_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/rangerkms_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.195.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.195.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 20:45:51 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901951, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 20:45:51.279601 14093 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 20:45:51.289907 14093 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 20:45:52.156024 14093 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 20:45:52.156096 14093 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms
I20260619 20:45:52.156109 14093 mini_ranger_kms.cc:227] Using postgres at 127.13.195.84:46065
2026-06-19 20:45:52,397 [I] DB FLAVOR :POSTGRES
2026-06-19 20:45:52,397 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:45:52,397 [I] Checking connection
2026-06-19 20:45:52,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 20:45:52,669 [I] connection success
2026-06-19 20:45:52,669 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 20:45:52,669 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 20:45:52,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 20:45:52,914 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 20:45:52,914 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:45:52,915 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 20:45:52,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:46065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 20:45:53,188 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 20:45:53.691272 14093 mini_ranger_kms.cc:310] Created kms service
I20260619 20:45:53.908241 14093 mini_ranger_kms.cc:326] Created kudu user
I20260619 20:45:53.950882 14093 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 20:45:53 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901951, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:45:54.328820 14093 mini_ranger_kms.cc:384] Added ranger policy
I20260619 20:45:54.328943 14093 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms:/tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 20:45:54.328974 14093 mini_ranger_kms.cc:240] Using host: 127.13.195.84
I20260619 20:45:54.333149 14093 mini_ranger_kms.cc:276] Ranger KMS PID: 25284
I20260619 20:45:54.333312 14093 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.195.84:40699
Jun 19, 2026 8:45:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:45:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:45:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:45:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:45:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:45:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 20:45:56.639070 14093 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.359s user 0.000s sys 0.212s
I20260619 20:45:56.639191 14093 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 20:45:56.639276 14093 mini_ranger_kms.cc:401] 127.13.195.84:40699/kms/v1/keys
Jun 19 20:45:56 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901956, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.195.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 20:45:57 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901956, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.195.84@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
Jun 19 20:45:57 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901956, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.195.84@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
Jun 19 20:45:59 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:45:59.663316 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42983
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:42983
--ranger_config_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:40699/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:45:59.753857 25335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:45:59.754066 25335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:45:59.754088 25335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:45:59.755213 25335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:45:59.755249 25335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:45:59.755263 25335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:45:59.755276 25335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:45:59.755288 25335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:45:59.756767 25335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.195.84:40699/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:42983
--ranger_config_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:45:59.756946 25335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:45:59.757121 25335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:45:59.762326 25340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:59.762410 25335 server_base.cc:1034] running on GCE node
W20260619 20:45:59.762423 25343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:45:59.762362 25341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:45:59.762903 25335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:45:59.763262 25335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:45:59.764442 25335 hybrid_clock.cc:648] HybridClock initialized: now 1781901959764428 us; error 44 us; skew 500 ppm
Jun 19 20:45:59 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901959, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:45:59.770220 25335 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:45:59.770713 25335 webserver.cc:460] Webserver started at http://127.13.195.126:45215/ using document root <none> and password file <none>
I20260619 20:45:59.770907 25335 fs_manager.cc:359] Metadata directory not provided
I20260619 20:45:59.770944 25335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:45:59.771035 25335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:45:59 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901959, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:45:59.828445 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "b7edf5b5a29e4672a31e68999d4a7c96"
format_stamp: "Formatted at 2026-06-19 20:45:59 on dist-test-slave-n2td"
server_key: "6eb6171d237633b74553cd2874381e59"
server_key_iv: "ecb246b87d8f53dfad6a335e388a1833"
server_key_version: "kuduclusterkey@0"
I20260619 20:45:59.828867 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "b7edf5b5a29e4672a31e68999d4a7c96"
format_stamp: "Formatted at 2026-06-19 20:45:59 on dist-test-slave-n2td"
server_key: "6eb6171d237633b74553cd2874381e59"
server_key_iv: "ecb246b87d8f53dfad6a335e388a1833"
server_key_version: "kuduclusterkey@0"
I20260619 20:45:59.830926 25335 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 20:45:59.845974 25350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:45:59.846258 25335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:45:59.846423 25335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "b7edf5b5a29e4672a31e68999d4a7c96"
format_stamp: "Formatted at 2026-06-19 20:45:59 on dist-test-slave-n2td"
server_key: "6eb6171d237633b74553cd2874381e59"
server_key_iv: "ecb246b87d8f53dfad6a335e388a1833"
server_key_version: "kuduclusterkey@0"
I20260619 20:45:59.846560 25335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:45:59.861655 25335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:45:59.863823 25335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:45:59.863993 25335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:45:59.868975 25335 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:42983
I20260619 20:45:59.869016 25402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:42983 every 8 connection(s)
I20260619 20:45:59.869375 25335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:45:59.870153 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 25335
I20260619 20:45:59.870265 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:45:59.870468 14093 external_mini_cluster.cc:1428] Setting key 449c3d37095c199d6f79e7025e123473
I20260619 20:45:59.870692 25403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:45:59.873991 25403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96: Bootstrap starting.
I20260619 20:45:59.874670 25403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96: Neither blocks nor log segments found. Creating new log.
I20260619 20:45:59.874977 25403 log.cc:824] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96: Log is configured to *not* fsync() on all Append() calls
I20260619 20:45:59.875788 25403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96: No bootstrap required, opened a new log
Jun 19 20:45:59 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:45:59.878721 25403 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42983 } }
I20260619 20:45:59.878944 25403 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:45:59.879001 25403 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7edf5b5a29e4672a31e68999d4a7c96, State: Initialized, Role: FOLLOWER
I20260619 20:45:59.879154 25403 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42983 } }
I20260619 20:45:59.879253 25403 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:45:59.879307 25403 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:45:59.879374 25403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:45:59.880084 25403 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42983 } }
I20260619 20:45:59.880204 25403 leader_election.cc:302] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7edf5b5a29e4672a31e68999d4a7c96; no voters:
I20260619 20:45:59.880376 25403 leader_election.cc:288] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:45:59.880427 25408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:45:59.880605 25408 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [term 1 LEADER]: Becoming Leader. State: Replica: b7edf5b5a29e4672a31e68999d4a7c96, State: Running, Role: LEADER
I20260619 20:45:59.880719 25403 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:45:59.880728 25408 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42983 } }
I20260619 20:45:59.882601 25409 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7edf5b5a29e4672a31e68999d4a7c96. Latest consensus state: current_term: 1 leader_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42983 } } }
I20260619 20:45:59.882778 25409 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [sys.catalog]: This master's current role is: LEADER
I20260619 20:45:59.882848 25403 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 20:45:59.883095 25422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:45:59.883451 25410 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7edf5b5a29e4672a31e68999d4a7c96" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 42983 } } }
I20260619 20:45:59.883597 25410 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96 [sys.catalog]: This master's current role is: LEADER
I20260619 20:45:59.883692 25406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:45:59.871015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57114 (local address 127.13.195.126:42983)
0619 20:45:59.871220 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 20:45:59.871229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:45:59.871261 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:45:59.871946 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:45:59.871956 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:45:59.872037 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:45:59.872189 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:45:59.872970 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:59.874570 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:59.875546 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:45:59.876204 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:45:59.876322 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:45:59.879359 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:45:59.879363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:45:59.879363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:45:59.879376 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:45:59.881402 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:59.881861 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:59.881865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:59.881866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:59.881969 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:45:59.882241 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:45:59.882242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:45:59.882242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:45:59.882725 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:45:59.882923 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 20:45:59.883238 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 20:45:59.883382 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260619 20:45:59.883589 25422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:45:59.890563 25422 catalog_manager.cc:1347] Generated new cluster ID: 3587b768e9fd42b3aa7180ca081f4c5e
I20260619 20:45:59.890812 25422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:45:59.890717 25426 mvcc.cc:205] Tried to move back new op lower bound from 7298670427703902208 to 7298670427671834624. Current Snapshot: MvccSnapshot[applied={T|T < 7298670427703902208 or (T in {7298670427703902208})}]
I20260619 20:45:59.912667 25422 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:45:59.913656 25422 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:45:59.919512 25422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96: Generated new TSK 0
I20260619 20:45:59.919754 25422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:46:00.651748 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:40699/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42983
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:00.772946 25448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:00.773247 25448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:00.773284 25448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:00.774796 25448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:00.774843 25448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:00.774873 25448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:46:00.776436 25448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.195.84:40699/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:00.776701 25448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:00.776962 25448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:00.782582 25453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:00.782582 25456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:00.782927 25454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:00.782910 25448 server_base.cc:1034] running on GCE node
I20260619 20:46:00.783267 25448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:00.783543 25448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:00.784744 25448 hybrid_clock.cc:648] HybridClock initialized: now 1781901960784734 us; error 27 us; skew 500 ppm
Jun 19 20:46:00 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901960, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:00.787724 25448 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:46:00.788276 25448 webserver.cc:460] Webserver started at http://127.13.195.65:41943/ using document root <none> and password file <none>
I20260619 20:46:00.788534 25448 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:00.788605 25448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:00.788746 25448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:46:00 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901960, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:00.802739 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "09e9b9877e7f449db1d76eba4b7e5162"
format_stamp: "Formatted at 2026-06-19 20:46:00 on dist-test-slave-n2td"
server_key: "d8c8385be679a6a1588bd3b8a40b8b97"
server_key_iv: "e63e8a8d83100c88f1661daea598f3d0"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:00.803340 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "09e9b9877e7f449db1d76eba4b7e5162"
format_stamp: "Formatted at 2026-06-19 20:46:00 on dist-test-slave-n2td"
server_key: "d8c8385be679a6a1588bd3b8a40b8b97"
server_key_iv: "e63e8a8d83100c88f1661daea598f3d0"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:00.805212 25448 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:46:00.822640 25465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:00.822903 25448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 20:46:00.822997 25448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "09e9b9877e7f449db1d76eba4b7e5162"
format_stamp: "Formatted at 2026-06-19 20:46:00 on dist-test-slave-n2td"
server_key: "d8c8385be679a6a1588bd3b8a40b8b97"
server_key_iv: "e63e8a8d83100c88f1661daea598f3d0"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:00.823107 25448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:00.859422 25448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:00.861559 25448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:00.861729 25448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:00.861951 25448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:00.862329 25448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:00.862390 25448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:00.862432 25448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:00.862480 25448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:00.872509 25448 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:36529
I20260619 20:46:00.872951 25448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:46:00.874614 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 25448
I20260619 20:46:00.874727 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:46:00.874923 14093 external_mini_cluster.cc:1428] Setting key f2e21271cc538c8b72a1f9928e21a1bd
I20260619 20:46:00.877315 25578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:36529 every 8 connection(s)
Jun 19 20:46:00 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901960, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:00.885912 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:00.873693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42983 (local address 127.13.195.65:35535)
0619 20:46:00.873843 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:00.873887 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:00.874572 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:00.874860 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:00.874864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:00.875124 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:00.875655 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:00.875662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:00.876363 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:00.876364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:00.877453 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:00.877455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:00.878028 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:00.878029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:00.878146 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:00.879942 (+ 1796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:00.879952 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:00.881542 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:00.884061 (+ 2519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:00.884069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:00.884071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:00.884398 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:00.884658 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:00.884659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:00.884659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:00.884818 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:00.885149 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:00.885156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:00.885341 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 20:46:00.885461 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 20:46:00.885643 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260619 20:46:00.886400 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:00.873684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:35535 (local address 127.13.195.126:42983)
0619 20:46:00.874534 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:00.874537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:00.874548 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:46:00.874698 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:00.874699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:00.874712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:00.874787 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:00.875756 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:00.876270 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:00.877571 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:00.877896 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:00.877958 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:00.881648 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:00.881650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:00.881650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:00.881657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:00.883953 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:00.884511 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:00.884512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:00.884512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:00.884582 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:00.884898 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:00.884899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:00.884899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:00.885052 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:00.885120 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:00.886188 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0619 20:46:00.886283 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":79,"threads_started":1}
I20260619 20:46:00.887099 25579 heartbeater.cc:347] Connected to a master server at 127.13.195.126:42983
I20260619 20:46:00.887199 25579 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:00.887399 25579 heartbeater.cc:511] Master 127.13.195.126:42983 requested a full tablet report, sending...
I20260619 20:46:00.888031 25367 ts_manager.cc:194] Registered new tserver with Master: 09e9b9877e7f449db1d76eba4b7e5162 (127.13.195.65:36529)
I20260619 20:46:00.889025 25367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:35535
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:46:00.916895 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:40699/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42983
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 20:46:01.012878 25587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:01.013105 25587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:01.013195 25587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:01.014423 25587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:01.014539 25587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:01.014622 25587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:46:01.016626 25587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.195.84:40699/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:01.017019 25587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:01.017346 25587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:01.023545 25593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:01.023965 25587 server_base.cc:1034] running on GCE node
W20260619 20:46:01.024197 25592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:01.024505 25595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:01.024796 25587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:01.025089 25587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:01.026288 25587 hybrid_clock.cc:648] HybridClock initialized: now 1781901961026265 us; error 35 us; skew 500 ppm
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:01.030325 25587 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:46:01.030972 25587 webserver.cc:460] Webserver started at http://127.13.195.66:45867/ using document root <none> and password file <none>
I20260619 20:46:01.031319 25587 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:01.031451 25587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:01.031646 25587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:01.047652 25587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "98f349ffd14247af9b7b728d0e153a6d"
format_stamp: "Formatted at 2026-06-19 20:46:01 on dist-test-slave-n2td"
server_key: "f28d4595d2fde9daff5e3a9011615a41"
server_key_iv: "41c4898c4f7db51dd6dc6593f8580535"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:01.048238 25587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "98f349ffd14247af9b7b728d0e153a6d"
format_stamp: "Formatted at 2026-06-19 20:46:01 on dist-test-slave-n2td"
server_key: "f28d4595d2fde9daff5e3a9011615a41"
server_key_iv: "41c4898c4f7db51dd6dc6593f8580535"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:01.050336 25587 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 20:46:01.064848 25602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:01.065052 25587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260619 20:46:01.065157 25587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "98f349ffd14247af9b7b728d0e153a6d"
format_stamp: "Formatted at 2026-06-19 20:46:01 on dist-test-slave-n2td"
server_key: "f28d4595d2fde9daff5e3a9011615a41"
server_key_iv: "41c4898c4f7db51dd6dc6593f8580535"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:01.065263 25587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:01.073993 25587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:01.076155 25587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:01.076308 25587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:01.076541 25587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:01.076853 25587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:01.076905 25587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:01.076947 25587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:01.076974 25587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:01.083993 25587 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:45587
I20260619 20:46:01.084077 25715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:45587 every 8 connection(s)
I20260619 20:46:01.084399 25587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:46:01.085016 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 25587
I20260619 20:46:01.085142 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:46:01.085366 14093 external_mini_cluster.cc:1428] Setting key d8a76fbff8d7c3f0d57410ba3b4b706b
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
I20260619 20:46:01.102496 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.085494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:47821 (local address 127.13.195.126:42983)
0619 20:46:01.085556 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.085559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.085787 (+ 228us) server_negotiation.cc:407] Connection header received
0619 20:46:01.086526 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.086527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.086542 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.086598 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:01.087413 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.089870 (+ 2457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.092022 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.092372 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.092422 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.095247 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:01.095248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:01.095249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:01.095257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:01.097245 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:01.100673 (+ 3428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:01.100674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:01.100675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:01.100756 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:01.101122 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:01.101123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:01.101123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:01.101286 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:01.101346 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.102213 (+ 867us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.102325 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:46:01.103360 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.085207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42983 (local address 127.13.195.66:47821)
0619 20:46:01.085483 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.085680 (+ 197us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.086361 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:01.086678 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:01.086682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:01.086956 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:01.087270 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.087302 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.090018 (+ 2716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.090019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.091886 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.091890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.092528 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.092529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.092699 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.093298 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:01.093309 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:01.095125 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:01.097346 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:01.097354 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:01.097356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:01.100562 (+ 3206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:01.100849 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:01.100850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:01.100850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:01.101028 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:01.101800 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:01.101808 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:01.102022 (+ 214us) client_negotiation.cc:769] Sending connection context
0619 20:46:01.102883 (+ 861us) client_negotiation.cc:240] Negotiation successful
0619 20:46:01.103092 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260619 20:46:01.104136 25716 heartbeater.cc:347] Connected to a master server at 127.13.195.126:42983
I20260619 20:46:01.104249 25716 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:01.104457 25716 heartbeater.cc:511] Master 127.13.195.126:42983 requested a full tablet report, sending...
I20260619 20:46:01.104890 25367 ts_manager.cc:194] Registered new tserver with Master: 98f349ffd14247af9b7b728d0e153a6d (127.13.195.66:45587)
I20260619 20:46:01.105472 25367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:47821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:46:01.117655 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:40699/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42983
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:01.245375 25722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:01.245714 25722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:01.245769 25722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:01.247598 25722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:01.247666 25722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:01.247711 25722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:46:01.250046 25722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.195.84:40699/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:42983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:01.250329 25722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:01.250710 25722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:01.257346 25730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:01.257359 25727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:01.257822 25728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:01.258657 25722 server_base.cc:1034] running on GCE node
I20260619 20:46:01.258874 25722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:01.259176 25722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:01.260357 25722 hybrid_clock.cc:648] HybridClock initialized: now 1781901961260329 us; error 43 us; skew 500 ppm
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:01.263787 25722 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:46:01.264390 25722 webserver.cc:460] Webserver started at http://127.13.195.67:35063/ using document root <none> and password file <none>
I20260619 20:46:01.264690 25722 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:01.264766 25722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:01.264967 25722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:01.280436 25722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "affdd3cb177e4b71aa244e6b01f5df14"
format_stamp: "Formatted at 2026-06-19 20:46:01 on dist-test-slave-n2td"
server_key: "e003098a4cd9e8a5ec0c7279bd7825d9"
server_key_iv: "2143eb4b44b8b7882390a0aeec3978f1"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:01.281003 25722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "affdd3cb177e4b71aa244e6b01f5df14"
format_stamp: "Formatted at 2026-06-19 20:46:01 on dist-test-slave-n2td"
server_key: "e003098a4cd9e8a5ec0c7279bd7825d9"
server_key_iv: "2143eb4b44b8b7882390a0aeec3978f1"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:01.292603 25722 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.002s
I20260619 20:46:01.310559 25737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:01.310830 25722 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260619 20:46:01.310935 25722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "affdd3cb177e4b71aa244e6b01f5df14"
format_stamp: "Formatted at 2026-06-19 20:46:01 on dist-test-slave-n2td"
server_key: "e003098a4cd9e8a5ec0c7279bd7825d9"
server_key_iv: "2143eb4b44b8b7882390a0aeec3978f1"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:01.311033 25722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:01.326903 25722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:01.329370 25722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:01.329557 25722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:01.329803 25722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:01.330124 25722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:01.330180 25722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:01.330227 25722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:01.330261 25722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:01.337054 25722 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:44843
I20260619 20:46:01.337214 25850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:44843 every 8 connection(s)
I20260619 20:46:01.337509 25722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:01.344724 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 25722
I20260619 20:46:01.344863 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:46:01.345094 14093 external_mini_cluster.cc:1428] Setting key ca2923a066f3c28fc626585397520ff3
I20260619 20:46:01.348234 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.338318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:35347 (local address 127.13.195.126:42983)
0619 20:46:01.338389 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.338392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.338612 (+ 220us) server_negotiation.cc:407] Connection header received
0619 20:46:01.339074 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.339075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.339088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.339169 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:01.339712 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.340461 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.341515 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.341865 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.341942 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.344488 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:01.344489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:01.344490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:01.344497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:01.346140 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:01.346736 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:01.346737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:01.346738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:01.346789 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:01.347517 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:01.347518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:01.347518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:01.347634 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:01.347686 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.348087 (+ 401us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.348151 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:46:01.348531 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.338240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:42983 (local address 127.13.195.67:35347)
0619 20:46:01.338474 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.338516 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.338980 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:01.339237 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:01.339241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:01.339424 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:01.339619 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.339623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.340594 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.340595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.341424 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.341426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.341948 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.341948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.342061 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.342948 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:01.342960 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:01.344391 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:01.346242 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:01.346249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:01.346252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:01.346648 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:01.346861 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:01.346862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:01.346862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:01.347431 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:01.347712 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:01.347720 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:01.347916 (+ 196us) client_negotiation.cc:769] Sending connection context
0619 20:46:01.348044 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 20:46:01.348236 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
I20260619 20:46:01.349071 25851 heartbeater.cc:347] Connected to a master server at 127.13.195.126:42983
I20260619 20:46:01.349184 25851 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:01.349406 25851 heartbeater.cc:511] Master 127.13.195.126:42983 requested a full tablet report, sending...
I20260619 20:46:01.350283 25367 ts_manager.cc:194] Registered new tserver with Master: affdd3cb177e4b71aa244e6b01f5df14 (127.13.195.67:44843)
I20260619 20:46:01.350798 25367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:35347
I20260619 20:46:01.364666 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:46:01.373643 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.366234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.13.195.126:42983)
0619 20:46:01.366282 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.366284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.366418 (+ 134us) server_negotiation.cc:407] Connection header received
0619 20:46:01.366563 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.366564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.366575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.366629 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:01.367250 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.367795 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.368634 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.368882 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.368940 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.370540 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:01.370542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:01.370542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:01.370550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:01.372311 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:01.372739 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:01.372740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:01.372741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:01.372800 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:01.373005 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:01.373006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:01.373007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:01.373142 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:01.373224 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.373450 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.373537 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:46:01.376014 25367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:46:01.377712 25367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:46:01.386659 25864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.380659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:44843 (local address 127.0.0.1:45258)
0619 20:46:01.381601 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.381644 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.381746 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:01.382028 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:01.382031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:01.382033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:01.382113 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.382115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.383702 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.383703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.385128 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.385130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.386315 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.386316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.386399 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.386413 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:46:01.386510 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 20:46:01.386569 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":20,"thread_start_us":103,"threads_started":1}
I20260619 20:46:01.387238 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.381143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.13.195.67:44843)
0619 20:46:01.381429 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.381434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.381727 (+ 293us) server_negotiation.cc:407] Connection header received
0619 20:46:01.381835 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.381838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.381855 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.381934 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:01.382222 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.383614 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.385229 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.386124 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.386197 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.386282 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.387107 (+ 825us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.387152 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260619 20:46:01.388082 25785 tablet_service.cc:1467] Processing CreateTablet for tablet 6b33c30bdaa44a7d8083b2dd5548e967 (DEFAULT_TABLE table=test-table [id=690fdc545bd645f6bdf4e39c0a672ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:01.388489 25785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b33c30bdaa44a7d8083b2dd5548e967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:01.389150 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.380915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:36529 (local address 127.0.0.1:51444)
0619 20:46:01.381980 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.381988 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.382063 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:01.382737 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:01.382737 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:01.382739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:01.382771 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.382772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.384047 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.384048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.385094 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.385097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.388889 (+ 3792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.388890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.388977 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.388982 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:46:01.389030 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:46:01.389079 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"thread_start_us":63,"threads_started":1}
I20260619 20:46:01.389424 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.381258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.13.195.65:36529)
0619 20:46:01.381530 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.381534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.382039 (+ 505us) server_negotiation.cc:407] Connection header received
0619 20:46:01.382232 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.382234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.382252 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.382321 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:01.382837 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.383948 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.385228 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.386124 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.386203 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.386277 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.389312 (+ 3035us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.389359 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":58,"threads_started":1}
I20260619 20:46:01.390091 25513 tablet_service.cc:1467] Processing CreateTablet for tablet 6b33c30bdaa44a7d8083b2dd5548e967 (DEFAULT_TABLE table=test-table [id=690fdc545bd645f6bdf4e39c0a672ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:01.390475 25513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b33c30bdaa44a7d8083b2dd5548e967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:01.391665 25869 tablet_bootstrap.cc:492] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Bootstrap starting.
I20260619 20:46:01.392675 25869 tablet_bootstrap.cc:654] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:01.392971 25869 log.cc:824] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:01.393908 25869 tablet_bootstrap.cc:492] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: No bootstrap required, opened a new log
I20260619 20:46:01.394021 25869 ts_tablet_manager.cc:1397] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:46:01.394601 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.380659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:45587 (local address 127.0.0.1:51258)
0619 20:46:01.381865 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.381874 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.382109 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:01.382629 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:01.382630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:01.382632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:01.382681 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.382682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.387552 (+ 4870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.387553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.388789 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.388791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.394284 (+ 5493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.394284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.394360 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.394367 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:46:01.394475 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 20:46:01.394522 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1190,"mutex_wait_us":5,"thread_start_us":97,"threads_started":1}
I20260619 20:46:01.394937 25866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.381044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.13.195.66:45587)
0619 20:46:01.381208 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.381213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.381931 (+ 718us) server_negotiation.cc:407] Connection header received
0619 20:46:01.382461 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.382464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.382481 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.382533 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:01.386343 (+ 3810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.387447 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.388913 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.394188 (+ 5275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.394275 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.394362 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.394793 (+ 431us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.394851 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260619 20:46:01.396093 25650 tablet_service.cc:1467] Processing CreateTablet for tablet 6b33c30bdaa44a7d8083b2dd5548e967 (DEFAULT_TABLE table=test-table [id=690fdc545bd645f6bdf4e39c0a672ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:01.396454 25650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b33c30bdaa44a7d8083b2dd5548e967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:01.397018 25869 raft_consensus.cc:348] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.397194 25869 raft_consensus.cc:374] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:01.397229 25869 raft_consensus.cc:729] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: affdd3cb177e4b71aa244e6b01f5df14, State: Initialized, Role: FOLLOWER
I20260619 20:46:01.397352 25869 consensus_queue.cc:260] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.397621 25869 ts_tablet_manager.cc:1428] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 20:46:01.397737 25851 heartbeater.cc:503] Master 127.13.195.126:42983 was elected leader, sending a full tablet report...
I20260619 20:46:01.398722 25871 tablet_bootstrap.cc:492] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162: Bootstrap starting.
I20260619 20:46:01.399426 25871 tablet_bootstrap.cc:654] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:01.399722 25871 log.cc:824] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:01.400988 25871 tablet_bootstrap.cc:492] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162: No bootstrap required, opened a new log
I20260619 20:46:01.401077 25871 ts_tablet_manager.cc:1397] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:46:01.402626 25873 tablet_bootstrap.cc:492] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Bootstrap starting.
I20260619 20:46:01.403770 25873 tablet_bootstrap.cc:654] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:01.404212 25873 log.cc:824] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:01.404106 25871 raft_consensus.cc:348] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.404292 25871 raft_consensus.cc:374] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:01.404369 25871 raft_consensus.cc:729] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e9b9877e7f449db1d76eba4b7e5162, State: Initialized, Role: FOLLOWER
I20260619 20:46:01.404567 25871 consensus_queue.cc:260] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.404872 25871 ts_tablet_manager.cc:1428] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 20:46:01.404897 25579 heartbeater.cc:503] Master 127.13.195.126:42983 was elected leader, sending a full tablet report...
I20260619 20:46:01.406994 25873 tablet_bootstrap.cc:492] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: No bootstrap required, opened a new log
I20260619 20:46:01.407088 25873 ts_tablet_manager.cc:1397] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 20:46:01.408484 25873 raft_consensus.cc:348] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.408646 25873 raft_consensus.cc:374] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:01.408699 25873 raft_consensus.cc:729] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f349ffd14247af9b7b728d0e153a6d, State: Initialized, Role: FOLLOWER
I20260619 20:46:01.408833 25873 consensus_queue.cc:260] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.409160 25873 ts_tablet_manager.cc:1428] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:46:01.409183 25716 heartbeater.cc:503] Master 127.13.195.126:42983 was elected leader, sending a full tablet report...
Jun 19 20:46:01 dist-test-slave-n2td krb5kdc[24891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:01.574663 25877 raft_consensus.cc:482] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:46:01.574821 25877 raft_consensus.cc:504] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.575245 25877 leader_election.cc:288] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers affdd3cb177e4b71aa244e6b01f5df14 (127.13.195.67:44843), 09e9b9877e7f449db1d76eba4b7e5162 (127.13.195.65:36529)
I20260619 20:46:01.579236 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.575706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:49251 (local address 127.13.195.65:36529)
0619 20:46:01.575929 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.575931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.575940 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:46:01.575964 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.575966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.575977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.576022 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:01.576349 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.577026 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.578306 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.578876 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.578923 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.578931 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.579135 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.579169 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260619 20:46:01.579252 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.575428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:44843 (local address 127.13.195.66:42311)
0619 20:46:01.575543 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.575555 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.575601 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:01.575882 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:01.575883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:01.575885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:01.575956 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.575957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.576956 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.576957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.578153 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.578156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.578980 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.578981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.579052 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.579056 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:46:01.579112 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:46:01.579156 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 20:46:01.579587 25878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.575474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:36529 (local address 127.13.195.66:49251)
0619 20:46:01.575684 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.575693 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.575735 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:01.576100 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:01.576101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:01.576103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:01.576262 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.576265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.577118 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.577119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.578206 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:01.578209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.578991 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.578992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:01.579062 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.579066 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:46:01.579461 (+ 395us) client_negotiation.cc:240] Negotiation successful
0619 20:46:01.579501 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":76,"threads_started":1}
I20260619 20:46:01.579699 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.575657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:42311 (local address 127.13.195.67:44843)
0619 20:46:01.575749 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.575751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.575760 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:46:01.575790 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.575791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.575801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.575868 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:01.576041 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.576799 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.578274 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.578868 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.578939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.578948 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.579587 (+ 639us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.579634 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 20:46:01.579969 25805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b33c30bdaa44a7d8083b2dd5548e967" candidate_uuid: "98f349ffd14247af9b7b728d0e153a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "affdd3cb177e4b71aa244e6b01f5df14" is_pre_election: true
I20260619 20:46:01.579955 25527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b33c30bdaa44a7d8083b2dd5548e967" candidate_uuid: "98f349ffd14247af9b7b728d0e153a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e9b9877e7f449db1d76eba4b7e5162" is_pre_election: true
I20260619 20:46:01.580122 25805 raft_consensus.cc:2454] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98f349ffd14247af9b7b728d0e153a6d in term 0.
I20260619 20:46:01.580123 25527 raft_consensus.cc:2454] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98f349ffd14247af9b7b728d0e153a6d in term 0.
I20260619 20:46:01.580364 25605 leader_election.cc:302] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e9b9877e7f449db1d76eba4b7e5162, 98f349ffd14247af9b7b728d0e153a6d; no voters:
I20260619 20:46:01.580498 25877 raft_consensus.cc:2781] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:46:01.580551 25877 raft_consensus.cc:482] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:46:01.580574 25877 raft_consensus.cc:3037] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:01.581535 25877 raft_consensus.cc:504] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.581672 25877 leader_election.cc:288] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [CANDIDATE]: Term 1 election: Requested vote from peers affdd3cb177e4b71aa244e6b01f5df14 (127.13.195.67:44843), 09e9b9877e7f449db1d76eba4b7e5162 (127.13.195.65:36529)
I20260619 20:46:01.581869 25805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b33c30bdaa44a7d8083b2dd5548e967" candidate_uuid: "98f349ffd14247af9b7b728d0e153a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "affdd3cb177e4b71aa244e6b01f5df14"
I20260619 20:46:01.581884 25527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b33c30bdaa44a7d8083b2dd5548e967" candidate_uuid: "98f349ffd14247af9b7b728d0e153a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e9b9877e7f449db1d76eba4b7e5162"
I20260619 20:46:01.581943 25805 raft_consensus.cc:3037] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:01.581951 25527 raft_consensus.cc:3037] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:01.582856 25805 raft_consensus.cc:2454] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f349ffd14247af9b7b728d0e153a6d in term 1.
I20260619 20:46:01.582918 25527 raft_consensus.cc:2454] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f349ffd14247af9b7b728d0e153a6d in term 1.
I20260619 20:46:01.583084 25603 leader_election.cc:302] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98f349ffd14247af9b7b728d0e153a6d, affdd3cb177e4b71aa244e6b01f5df14; no voters:
I20260619 20:46:01.583201 25877 raft_consensus.cc:2781] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:01.583357 25877 raft_consensus.cc:686] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 1 LEADER]: Becoming Leader. State: Replica: 98f349ffd14247af9b7b728d0e153a6d, State: Running, Role: LEADER
I20260619 20:46:01.583458 25877 consensus_queue.cc:237] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } }
I20260619 20:46:01.584296 25366 catalog_manager.cc:5495] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 98f349ffd14247af9b7b728d0e153a6d (127.13.195.66). New cstate: current_term: 1 leader_uuid: "98f349ffd14247af9b7b728d0e153a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f349ffd14247af9b7b728d0e153a6d" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 } health_report { overall_health: UNKNOWN } } }
W20260619 20:46:01.585157 25717 tablet.cc:2369] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:46:01.588236 25852 tablet.cc:2369] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:46:01.616132 25866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:01.612749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.13.195.66:45587)
0619 20:46:01.612793 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:01.612796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:01.612805 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:46:01.612847 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:01.612848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:01.612871 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:01.612958 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:46:01.613129 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.613901 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.615159 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:01.615420 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:01.615494 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:01.615716 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:46:01.615850 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:46:01.615968 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:01.616022 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 20:46:01.616048 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:46:01.618074 25805 raft_consensus.cc:1261] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 1 FOLLOWER]: Refusing update from remote peer 98f349ffd14247af9b7b728d0e153a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:01.618093 25527 raft_consensus.cc:1261] T 6b33c30bdaa44a7d8083b2dd5548e967 P 09e9b9877e7f449db1d76eba4b7e5162 [term 1 FOLLOWER]: Refusing update from remote peer 98f349ffd14247af9b7b728d0e153a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:01.618479 25877 consensus_queue.cc:1035] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "affdd3cb177e4b71aa244e6b01f5df14" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:01.618629 25879 consensus_queue.cc:1035] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "09e9b9877e7f449db1d76eba4b7e5162" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 36529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:01.623798 25367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:01.623910 25367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:01.624634 25367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7edf5b5a29e4672a31e68999d4a7c96: Sending DeleteTablet for 3 replicas of tablet 6b33c30bdaa44a7d8083b2dd5548e967
I20260619 20:46:01.625000 25650 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b33c30bdaa44a7d8083b2dd5548e967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:01 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:51258
I20260619 20:46:01.625006 25785 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b33c30bdaa44a7d8083b2dd5548e967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:01 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:45258
I20260619 20:46:01.625015 25513 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b33c30bdaa44a7d8083b2dd5548e967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:01 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:51444
I20260619 20:46:01.625406 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 25448
I20260619 20:46:01.630975 25883 mvcc.cc:205] Tried to move back new op lower bound from 7298670434784768000 to 7298670434646577152. Current Snapshot: MvccSnapshot[applied={T|T < 7298670434784768000}]
I20260619 20:46:01.631124 25891 tablet_replica.cc:330] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: stopping tablet replica
I20260619 20:46:01.631278 25890 tablet_replica.cc:330] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: stopping tablet replica
I20260619 20:46:01.631541 25890 raft_consensus.cc:2229] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:46:01.631688 25890 raft_consensus.cc:2258] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:01.631867 25891 raft_consensus.cc:2229] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:46:01.632032 25891 raft_consensus.cc:2258] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:01.632293 25890 ts_tablet_manager.cc:1905] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:01.632478 25891 ts_tablet_manager.cc:1905] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:01.634537 25890 ts_tablet_manager.cc:1918] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:01.634611 25891 ts_tablet_manager.cc:1918] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:01.634697 25890 log.cc:1196] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/6b33c30bdaa44a7d8083b2dd5548e967
I20260619 20:46:01.634744 25891 log.cc:1196] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/6b33c30bdaa44a7d8083b2dd5548e967
I20260619 20:46:01.635172 25891 ts_tablet_manager.cc:1939] T 6b33c30bdaa44a7d8083b2dd5548e967 P 98f349ffd14247af9b7b728d0e153a6d: Deleting consensus metadata
I20260619 20:46:01.635442 25890 ts_tablet_manager.cc:1939] T 6b33c30bdaa44a7d8083b2dd5548e967 P affdd3cb177e4b71aa244e6b01f5df14: Deleting consensus metadata
I20260619 20:46:01.635710 25352 catalog_manager.cc:4841] TS 98f349ffd14247af9b7b728d0e153a6d (127.13.195.66:45587): tablet 6b33c30bdaa44a7d8083b2dd5548e967 (table test-table [id=690fdc545bd645f6bdf4e39c0a672ccc]) successfully deleted
W20260619 20:46:01.635787 25353 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:36529 (error 108)
I20260619 20:46:01.635908 25351 catalog_manager.cc:4841] TS affdd3cb177e4b71aa244e6b01f5df14 (127.13.195.67:44843): tablet 6b33c30bdaa44a7d8083b2dd5548e967 (table test-table [id=690fdc545bd645f6bdf4e39c0a672ccc]) successfully deleted
W20260619 20:46:01.636394 25863 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:46:01.636221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:36529 (local address 127.0.0.1:51456)
0619 20:46:01.636298 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:01.636310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:01.636349 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:36529: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 20:46:01.636543 25353 catalog_manager.cc:4584] TS 09e9b9877e7f449db1d76eba4b7e5162 (127.13.195.65:36529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b33c30bdaa44a7d8083b2dd5548e967: Network error: Client connection negotiation failed: client connection to 127.13.195.65:36529: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260619 20:46:01.638648 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 25587
I20260619 20:46:01.644094 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 25722
I20260619 20:46:01.648563 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 25335
I20260619 20:46:01.653956 14093 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 20:46:01.998678 14093 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 20:46:01.998781 14093 mini_ranger.cc:66] Stopping Ranger...
I20260619 20:46:02.343611 14093 mini_ranger.cc:68] Stopped Ranger
2026-06-19 20:46:02.343 UTC [24921] LOG: received smart shutdown request
2026-06-19 20:46:02.347 UTC [24921] LOG: background worker "logical replication launcher" (PID 24930) exited with exit code 1
2026-06-19 20:46:02.347 UTC [24925] LOG: shutting down
2026-06-19 20:46:02.520 UTC [24921] LOG: database system is shut down
2026-06-19T20:46:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47105 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:46:02 dist-test-slave-n2td krb5kdc[25903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:46:02 dist-test-slave-n2td krb5kdc[25903](info): set up 2 sockets
Jun 19 20:46:02 dist-test-slave-n2td krb5kdc[25903](info): commencing operation
krb5kdc: starting...
W20260619 20:46:03.665338 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:46:03 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:46:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:46:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/rangeradmin_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/rangeradmin_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/rangerlookup_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/rangerlookup_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
I20260619 20:46:03.828963 14093 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskhOhR3l/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/postgres -l logfile start
2026-06-19 20:46:06.291 UTC [25933] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 20:46:06.291 UTC [25933] LOG: listening on IPv4 address "127.13.195.84", port 58293
2026-06-19 20:46:06.294 UTC [25933] LOG: listening on Unix socket "/tmp/.s.PGSQL.58293"
2026-06-19 20:46:06.331 UTC [25936] LOG: database system was shut down at 2026-06-19 20:46:04 UTC
2026-06-19 20:46:06.335 UTC [25933] LOG: database system is ready to accept connections
I20260619 20:46:07.297243 14093 mini_postgres.cc:96] Postgres bound to 58293
2026-06-19 20:46:07.303 UTC [25944] FATAL: database "slave" does not exist
127.13.195.84:58293 - accepting connections
I20260619 20:46:07.303858 14093 mini_ranger.cc:161] Starting Ranger...
I20260619 20:46:07.318502 14093 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 20:46:07.789733 14093 mini_ranger.cc:87] Created ranger Postgres database
I20260619 20:46:07.789805 14093 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-admin
2026-06-19 20:46:08,084 [I] DB FLAVOR :POSTGRES
2026-06-19 20:46:08,084 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:46:08,085 [I] Checking connection..
2026-06-19 20:46:08,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 20:46:08,329 [I] Checking connection passed.
2026-06-19 20:46:08,329 [I] --------- Verifying version history table ---------
2026-06-19 20:46:08,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:46:08,562 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:46:08,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:46:08,823 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:46:08,823 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 20:46:08,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 20:46:09.062 UTC [26008] WARNING: there is no transaction in progress
2026-06-19 20:46:09,068 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 20:46:09,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:46:09,333 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 20:46:09,333 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:46:09,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 20:46:09,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 20:46:09,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-n2td.c.gcp-upstream.internal','N') ;"
2026-06-19 20:46:10,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:46:10,305 [I] Table x_portal_user does not exist in database ranger
2026-06-19 20:46:10,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:46:10,555 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 20:46:10,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:46:10,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:46:11,075 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 20:46:11,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 20:46:12.027 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.044 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.060 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.076 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.095 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.112 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.166 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.173 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.187 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.197 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.208 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.222 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.232 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.238 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.245 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12.252 UTC [26116] WARNING: there is no transaction in progress
2026-06-19 20:46:12,904 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 20:46:12,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 20:46:13,142 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 20:46:13,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:46:13,379 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 20:46:13,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:46:13,635 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 20:46:13,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:46:13,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:46:14,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-n2td.c.gcp-upstream.internal';"
2026-06-19 20:46:14,394 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 20:46:14,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 20:46:14,654 [I] DB_PATCHES have already been applied
I20260619 20:46:14.659813 14093 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-admin:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskhOhR3l/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 20:46:14.659883 14093 mini_ranger.cc:193] Using host: 127.13.195.84
I20260619 20:46:14.663026 14093 mini_ranger.cc:227] Ranger admin URL: http://127.13.195.84:57335
Jun 19, 2026 8:46:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:46:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:46:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:46:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:46:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:46:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:46:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 20:46:15.788586 14093 mini_ranger.cc:160] Time spent starting Ranger: real 8.485s user 0.000s sys 0.009s
Jun 19, 2026 8:46:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:46:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 8:46:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:46:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:46:35 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:46:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 20:46:36.482066 14093 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.195.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.195.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/rangerkms_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.195.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/rangerkms_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.195.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.195.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.195.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/HTTP_127.13.195.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 20:46:36 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901996, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 20:46:36.573447 14093 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 20:46:36.583928 14093 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 20:46:37.449216 14093 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 20:46:37.449309 14093 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms
I20260619 20:46:37.449328 14093 mini_ranger_kms.cc:227] Using postgres at 127.13.195.84:58293
2026-06-19 20:46:37,622 [I] DB FLAVOR :POSTGRES
2026-06-19 20:46:37,623 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:46:37,623 [I] Checking connection
2026-06-19 20:46:37,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 20:46:37,878 [I] connection success
2026-06-19 20:46:37,879 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 20:46:37,879 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 20:46:37,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 20:46:38,141 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 20:46:38,141 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:46:38,141 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 20:46:38,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.195.84:58293/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 20:46:38,392 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 20:46:38.897854 14093 mini_ranger_kms.cc:310] Created kms service
I20260619 20:46:39.079265 14093 mini_ranger_kms.cc:326] Created kudu user
I20260619 20:46:39.123751 14093 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 20:46:39 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901996, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:39.512586 14093 mini_ranger_kms.cc:384] Added ranger policy
I20260619 20:46:39.512715 14093 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms:/tmp/dist-test-taskhOhR3l/build/release/bin/postgresql.jar:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskhOhR3l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 20:46:39.512734 14093 mini_ranger_kms.cc:240] Using host: 127.13.195.84
I20260619 20:46:39.516646 14093 mini_ranger_kms.cc:276] Ranger KMS PID: 26296
I20260619 20:46:39.516736 14093 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.195.84:38235
Jun 19, 2026 8:46:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:46:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:46:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:46:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:46:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:46:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 20:46:41.791404 14093 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.218s user 0.000s sys 0.152s
I20260619 20:46:41.791512 14093 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 20:46:41.791582 14093 mini_ranger_kms.cc:401] 127.13.195.84:38235/kms/v1/keys
Jun 19 20:46:41 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781902001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.195.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 20:46:42 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.195.84@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
Jun 19 20:46:42 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.195.84@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
Jun 19 20:46:44 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:46:44.994767 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32779
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:32779
--ranger_config_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:45.076732 26347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:45.076973 26347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:45.077013 26347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:45.078213 26347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:45.078291 26347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:45.078317 26347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:45.078358 26347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:45.078387 26347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:45.079977 26347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.195.84:38235/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:32779
--ranger_config_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:45.080242 26347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:45.080470 26347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:45.085793 26352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:45.085845 26347 server_base.cc:1034] running on GCE node
W20260619 20:46:45.085851 26355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:45.085759 26353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:45.086270 26347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:45.086670 26347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:45.087831 26347 hybrid_clock.cc:648] HybridClock initialized: now 1781902005087815 us; error 35 us; skew 500 ppm
Jun 19 20:46:45 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:45.090586 26347 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:46:45.091032 26347 webserver.cc:460] Webserver started at http://127.13.195.126:37643/ using document root <none> and password file <none>
I20260619 20:46:45.091289 26347 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:45.091351 26347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:45.091503 26347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:46:45 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902005, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:45.150476 26347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "369212b714dc43dfb18517bf7c1899d8"
format_stamp: "Formatted at 2026-06-19 20:46:45 on dist-test-slave-n2td"
server_key: "40eee0260c7680063cebff11062a0f39"
server_key_iv: "cd2e9cc8f239c42ecd7d99064389cab3"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:45.150949 26347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "369212b714dc43dfb18517bf7c1899d8"
format_stamp: "Formatted at 2026-06-19 20:46:45 on dist-test-slave-n2td"
server_key: "40eee0260c7680063cebff11062a0f39"
server_key_iv: "cd2e9cc8f239c42ecd7d99064389cab3"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:45.152453 26347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:45.167635 26362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:45.167827 26347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:46:45.167927 26347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "369212b714dc43dfb18517bf7c1899d8"
format_stamp: "Formatted at 2026-06-19 20:46:45 on dist-test-slave-n2td"
server_key: "40eee0260c7680063cebff11062a0f39"
server_key_iv: "cd2e9cc8f239c42ecd7d99064389cab3"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:45.168035 26347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:45.180212 26347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:45.182390 26347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:45.182607 26347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:45.187427 26347 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:32779
I20260619 20:46:45.187508 26414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:32779 every 8 connection(s)
I20260619 20:46:45.187855 26347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:46:45.188567 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:45.190716 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 26347
I20260619 20:46:45.190826 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:46:45.191025 14093 external_mini_cluster.cc:1428] Setting key 6ac4ca0c265caa2c16c1d53b2c002513
I20260619 20:46:45.191100 26415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Bootstrap starting.
I20260619 20:46:45.191859 26415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:45.192189 26415 log.cc:824] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:45.192996 26415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: No bootstrap required, opened a new log
I20260619 20:46:45.195150 26415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:45.195397 26415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:45.195511 26415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369212b714dc43dfb18517bf7c1899d8, State: Initialized, Role: FOLLOWER
I20260619 20:46:45.195693 26415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:45.195773 26415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:46:45.195803 26415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:46:45.195871 26415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:45.196441 26415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:45.196563 26415 leader_election.cc:302] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369212b714dc43dfb18517bf7c1899d8; no voters:
I20260619 20:46:45.196722 26415 leader_election.cc:288] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:46:45.196839 26420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:45.197021 26420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 LEADER]: Becoming Leader. State: Replica: 369212b714dc43dfb18517bf7c1899d8, State: Running, Role: LEADER
I20260619 20:46:45.197144 26415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:45.197153 26420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:45.197552 26420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "369212b714dc43dfb18517bf7c1899d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } } }
I20260619 20:46:45.197674 26420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:45.197822 26420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369212b714dc43dfb18517bf7c1899d8. Latest consensus state: current_term: 1 leader_uuid: "369212b714dc43dfb18517bf7c1899d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } } }
I20260619 20:46:45.197894 26420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:45.198577 26425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:46:45.199025 26415 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Jun 19 20:46:45 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:45.199642 26425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:46:45.206856 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:45.191571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.13.195.126:32779)
0619 20:46:45.191782 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:45.191791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:45.191824 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:46:45.192362 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:45.192371 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:45.192435 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:45.192599 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:45.193396 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:45.194877 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:45.195972 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:45.196612 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:45.196710 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:45.199625 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:45.199630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:45.199630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:45.199643 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:45.204949 (+ 5306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:45.205404 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:45.205407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:45.205407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:45.205520 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:45.205742 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:45.205743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:45.205743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:45.206144 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:45.206305 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:45.206465 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 20:46:45.206628 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
I20260619 20:46:45.219007 26425 catalog_manager.cc:1347] Generated new cluster ID: 04ff2eb40e3048f4b5284b7596083670
I20260619 20:46:45.219079 26425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:46:45.236552 26425 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:46:45.237371 26425 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:46:45.240715 26425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Generated new TSK 0
I20260619 20:46:45.240934 26425 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:46:45.959450 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32779
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:46.054365 26460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:46.054652 26460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:46.054692 26460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:46.055856 26460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:46.055920 26460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:46.055964 26460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:46:46.057439 26460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.195.84:38235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:46.057703 26460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:46.057940 26460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:46.062964 26465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.062952 26468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.062964 26466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:46.063567 26460 server_base.cc:1034] running on GCE node
I20260619 20:46:46.063817 26460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:46.064087 26460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:46.065274 26460 hybrid_clock.cc:648] HybridClock initialized: now 1781902006065259 us; error 32 us; skew 500 ppm
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:46.068018 26460 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:46:46.068495 26460 webserver.cc:460] Webserver started at http://127.13.195.65:33533/ using document root <none> and password file <none>
I20260619 20:46:46.068750 26460 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:46.068820 26460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:46.068972 26460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:46.083371 26460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "d3b92840a1374f9f8622514137b9428b"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "b23dbcdfb27553fcbf643866fad4a673"
server_key_iv: "9719852823d25402349142d303d189fe"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.083851 26460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "d3b92840a1374f9f8622514137b9428b"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "b23dbcdfb27553fcbf643866fad4a673"
server_key_iv: "9719852823d25402349142d303d189fe"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.085708 26460 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:46:46.099079 26475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.099303 26460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:46.099396 26460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "d3b92840a1374f9f8622514137b9428b"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "b23dbcdfb27553fcbf643866fad4a673"
server_key_iv: "9719852823d25402349142d303d189fe"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.099490 26460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:46.121001 26460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:46.123530 26460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:46.123696 26460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:46.123950 26460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:46.124305 26460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:46.124353 26460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.124384 26460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:46.124441 26460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.132154 26460 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:38901
I20260619 20:46:46.132236 26588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:38901 every 8 connection(s)
I20260619 20:46:46.132721 26460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:46.142299 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 26460
I20260619 20:46:46.142408 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:46:46.142730 14093 external_mini_cluster.cc:1428] Setting key 981796f5985f79d6954e124cd0fe8c59
I20260619 20:46:46.145777 26591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:46.133489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32779 (local address 127.13.195.65:39361)
0619 20:46:46.133693 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:46.133742 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:46.134354 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:46.134682 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:46.134686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:46.134925 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:46.135199 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:46.135205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.136211 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.136212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:46.137509 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:46.137512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.138072 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.138073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:46.138227 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:46.138971 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:46.138988 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:46.140443 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:46.142541 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:46.142547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:46.142550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:46.142859 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:46.143075 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:46.143076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:46.143077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:46.143198 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:46.143527 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:46.143533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:46.143701 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 20:46:46.145371 (+ 1670us) client_negotiation.cc:240] Negotiation successful
0619 20:46:46.145537 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260619 20:46:46.146337 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:46.133803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:39361 (local address 127.13.195.126:32779)
0619 20:46:46.133941 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:46.133943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:46.133954 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:46:46.134499 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:46.134501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:46.134517 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:46.134598 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:46.135318 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.136077 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.137615 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.137917 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.137987 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:46.140560 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:46.140562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:46.140562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:46.140571 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:46.142450 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:46.142944 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:46.142945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:46.142946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:46.143010 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:46.143274 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:46.143276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:46.143276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:46.143440 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:46.143502 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:46.146153 (+ 2651us) server_negotiation.cc:299] Negotiation successful
0619 20:46:46.146242 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260619 20:46:46.146919 26589 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32779
I20260619 20:46:46.147018 26589 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:46.147225 26589 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:46.147755 26379 ts_manager.cc:194] Registered new tserver with Master: d3b92840a1374f9f8622514137b9428b (127.13.195.65:38901)
I20260619 20:46:46.148636 26379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:39361
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:46:46.173343 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32779
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:46.291250 26597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:46.291556 26597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:46.291602 26597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:46.293426 26597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:46.293494 26597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:46.293540 26597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:46:46.295883 26597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.195.84:38235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:46.296139 26597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:46.296490 26597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:46.302798 26605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.302975 26607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.302832 26604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:46.304095 26597 server_base.cc:1034] running on GCE node
I20260619 20:46:46.304323 26597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:46.304617 26597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:46.305784 26597 hybrid_clock.cc:648] HybridClock initialized: now 1781902006305766 us; error 32 us; skew 500 ppm
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:46.309000 26597 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:46:46.309561 26597 webserver.cc:460] Webserver started at http://127.13.195.66:45849/ using document root <none> and password file <none>
I20260619 20:46:46.309844 26597 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:46.309911 26597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:46.310045 26597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:46.326886 26597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "39849a3169fe4b4dbc1c820e2597e4e7"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "bda1d28bfe8936eb02574abb0b960023"
server_key_iv: "b76e8728efb07a0647cea3d865c79be5"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.327358 26597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "39849a3169fe4b4dbc1c820e2597e4e7"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "bda1d28bfe8936eb02574abb0b960023"
server_key_iv: "b76e8728efb07a0647cea3d865c79be5"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.330597 26597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:46.343647 26614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.343825 26597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:46.343917 26597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "39849a3169fe4b4dbc1c820e2597e4e7"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "bda1d28bfe8936eb02574abb0b960023"
server_key_iv: "b76e8728efb07a0647cea3d865c79be5"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.344013 26597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:46.363540 26597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:46.365917 26597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:46.366106 26597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:46.366366 26597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:46.366834 26597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:46.366923 26597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.366997 26597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:46.367053 26597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.373469 26597 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:35469
I20260619 20:46:46.373972 26597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:46:46.376755 26727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:35469 every 8 connection(s)
I20260619 20:46:46.381424 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 26597
I20260619 20:46:46.381529 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:46:46.381726 14093 external_mini_cluster.cc:1428] Setting key 978bf8a1d4a31cc1287d609121bc2a09
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:46.390591 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:46.377600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:57643 (local address 127.13.195.126:32779)
0619 20:46:46.377660 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:46.377663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:46.378009 (+ 346us) server_negotiation.cc:407] Connection header received
0619 20:46:46.378735 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:46.378736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:46.378751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:46.378801 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:46.379768 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.380546 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.384368 (+ 3822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.384664 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.384724 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:46.387379 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:46.387381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:46.387381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:46.387388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:46.389095 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:46.389556 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:46.389557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:46.389557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:46.389612 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:46.389881 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:46.389882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:46.389882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:46.390005 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:46.390053 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:46.390385 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 20:46:46.390493 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:46:46.391321 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:46.377319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32779 (local address 127.13.195.66:57643)
0619 20:46:46.377878 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:46.377927 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:46.378609 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:46.379111 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:46.379115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:46.379364 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:46.379652 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:46.379658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.380677 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.380678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:46.384269 (+ 3591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:46.384271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.384828 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.384829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:46.384975 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:46.385588 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:46.385598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:46.387273 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:46.389193 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:46.389199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:46.389202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:46.389474 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:46.389692 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:46.389692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:46.389693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:46.389812 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:46.390082 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:46.390087 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:46.390240 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 20:46:46.390933 (+ 693us) client_negotiation.cc:240] Negotiation successful
0619 20:46:46.391091 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":91,"threads_started":1}
I20260619 20:46:46.391978 26728 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32779
I20260619 20:46:46.392081 26728 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:46.392290 26728 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:46.392716 26379 ts_manager.cc:194] Registered new tserver with Master: 39849a3169fe4b4dbc1c820e2597e4e7 (127.13.195.66:35469)
I20260619 20:46:46.393231 26379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:57643
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:46:46.421988 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32779
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:46.543463 26733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:46.543773 26733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:46.543817 26733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:46.545620 26733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:46.545686 26733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:46.545728 26733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:46:46.547909 26733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.195.84:38235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:46.548168 26733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:46.548478 26733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:46.554924 26742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:46.555640 26733 server_base.cc:1034] running on GCE node
W20260619 20:46:46.555684 26740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.555913 26739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:46.556206 26733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:46.556483 26733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:46.557667 26733 hybrid_clock.cc:648] HybridClock initialized: now 1781902006557638 us; error 42 us; skew 500 ppm
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:46.560948 26733 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:46:46.561448 26733 webserver.cc:460] Webserver started at http://127.13.195.67:42391/ using document root <none> and password file <none>
I20260619 20:46:46.561707 26733 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:46.561766 26733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:46.561895 26733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:46.584432 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "17b24e36dfd7446a9cd25339ccf5a96f"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "2b3012d03b513eeaa5f21de695c685b9"
server_key_iv: "00c72761d847fa8c898f9c5795d4da1c"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.584888 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "17b24e36dfd7446a9cd25339ccf5a96f"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "2b3012d03b513eeaa5f21de695c685b9"
server_key_iv: "00c72761d847fa8c898f9c5795d4da1c"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.592505 26733 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260619 20:46:46.609004 26749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.609200 26733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:46.609297 26733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "17b24e36dfd7446a9cd25339ccf5a96f"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "2b3012d03b513eeaa5f21de695c685b9"
server_key_iv: "00c72761d847fa8c898f9c5795d4da1c"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.609426 26733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:46.622653 26733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:46.624812 26733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:46.624969 26733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:46.625214 26733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:46.625582 26733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:46.625626 26733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.625656 26733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:46.625674 26733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.635960 26733 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:38257
I20260619 20:46:46.636415 26733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:46:46.637096 26862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:38257 every 8 connection(s)
I20260619 20:46:46.639164 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 26733
I20260619 20:46:46.639282 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:46:46.639477 14093 external_mini_cluster.cc:1428] Setting key 011a38fa117b14c08fd837ccbfecaf93
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:46.649958 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:46.638823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:36985 (local address 127.13.195.126:32779)
0619 20:46:46.638896 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:46.638899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:46.639420 (+ 521us) server_negotiation.cc:407] Connection header received
0619 20:46:46.640286 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:46.640288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:46.640303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:46.640334 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:46.641161 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.641938 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.643527 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.643831 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.643907 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:46.646518 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:46.646519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:46.646519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:46.646527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:46.648301 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:46.648834 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:46.648836 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:46.648836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:46.648904 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:46.649206 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:46.649207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:46.649207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:46.649353 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:46.649428 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:46.649771 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 20:46:46.649857 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:46:46.650134 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:46.638810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32779 (local address 127.13.195.67:36985)
0619 20:46:46.639286 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:46.639334 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:46.640159 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:46.640529 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:46.640534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:46.640776 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:46.641051 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:46.641058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.642058 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.642059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:46.643432 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:46.643435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:46.643945 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:46.643946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:46.644093 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:46.644731 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:46.644742 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:46.646389 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:46.648398 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:46.648406 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:46.648408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:46.648757 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:46.648980 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:46.648981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:46.648981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:46.649130 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:46.649419 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:46.649425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:46.649607 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 20:46:46.649730 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 20:46:46.649892 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":137,"threads_started":1}
I20260619 20:46:46.650902 26863 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32779
I20260619 20:46:46.651017 26863 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:46.651237 26863 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:46.651621 26378 ts_manager.cc:194] Registered new tserver with Master: 17b24e36dfd7446a9cd25339ccf5a96f (127.13.195.67:38257)
I20260619 20:46:46.652112 26378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:36985
I20260619 20:46:46.667462 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:46:46.668231 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 26460
I20260619 20:46:46.675209 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 26597
I20260619 20:46:46.682575 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 26733
I20260619 20:46:46.688146 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 26347
I20260619 20:46:46.692936 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32779
--webserver_interface=127.13.195.126
--webserver_port=37643
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:32779
--ranger_config_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:46.774843 26866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:46.775060 26866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:46.775082 26866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:46.776260 26866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:46.776307 26866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:46.776321 26866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:46.776335 26866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:46.776386 26866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:46.778038 26866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:32779
--ranger_config_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=37643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:46.778321 26866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:46.778579 26866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:46.783452 26872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.783465 26871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.783457 26874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:46.783668 26866 server_base.cc:1034] running on GCE node
I20260619 20:46:46.783934 26866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:46.784186 26866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:46.785337 26866 hybrid_clock.cc:648] HybridClock initialized: now 1781902006785323 us; error 42 us; skew 500 ppm
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:46.787752 26866 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:46:46.788223 26866 webserver.cc:460] Webserver started at http://127.13.195.126:37643/ using document root <none> and password file <none>
I20260619 20:46:46.788451 26866 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:46.788527 26866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:46.827903 26866 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:46.828737 26882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:46.828953 26866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:46.829046 26866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "369212b714dc43dfb18517bf7c1899d8"
format_stamp: "Formatted at 2026-06-19 20:46:45 on dist-test-slave-n2td"
server_key: "40eee0260c7680063cebff11062a0f39"
server_key_iv: "cd2e9cc8f239c42ecd7d99064389cab3"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:46.829443 26866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:46.840530 26866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:46.842675 26866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:46.842832 26866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:46.846982 26866 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:32779
I20260619 20:46:46.847024 26934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:32779 every 8 connection(s)
I20260619 20:46:46.847510 26866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:46:46.848217 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 26866
I20260619 20:46:46.848498 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:38901
--local_ip_for_outbound_sockets=127.13.195.65
--tserver_master_addrs=127.13.195.126:32779
--webserver_port=33533
--webserver_interface=127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:46:46.849217 26935 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:46:46.850066 26935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Bootstrap starting.
I20260619 20:46:46.860556 26935 log.cc:824] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:46.862828 26935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:46:46.863179 26935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Bootstrap complete.
I20260619 20:46:46.865888 26935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:46.866119 26935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369212b714dc43dfb18517bf7c1899d8, State: Initialized, Role: FOLLOWER
I20260619 20:46:46.866305 26935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:46.866374 26935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:46:46.866425 26935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:46:46.866488 26935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:46:46.867496 26935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:46.867581 26935 leader_election.cc:302] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369212b714dc43dfb18517bf7c1899d8; no voters:
I20260619 20:46:46.867797 26935 leader_election.cc:288] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:46:46.867838 26939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:46:46.867959 26939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [term 2 LEADER]: Becoming Leader. State: Replica: 369212b714dc43dfb18517bf7c1899d8, State: Running, Role: LEADER
I20260619 20:46:46.868063 26935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:46.868057 26939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } }
I20260619 20:46:46.868517 26941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369212b714dc43dfb18517bf7c1899d8. Latest consensus state: current_term: 2 leader_uuid: "369212b714dc43dfb18517bf7c1899d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } } }
I20260619 20:46:46.868597 26941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:46.868954 26944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:46:46.869413 26944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:46:46.869542 26944 catalog_manager.cc:1259] Loaded cluster ID: 04ff2eb40e3048f4b5284b7596083670
I20260619 20:46:46.869606 26944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:46:46.869827 26940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "369212b714dc43dfb18517bf7c1899d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369212b714dc43dfb18517bf7c1899d8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32779 } } }
I20260619 20:46:46.869892 26935 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 20:46:46.869989 26940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:46.873755 26944 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:46:46.874094 26944 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Loaded TSK: 0
I20260619 20:46:46.874358 26944 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:46:46.961707 26937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:46.961973 26937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:46.962009 26937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:46.963677 26937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:46.963737 26937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:46.963779 26937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:46:46.965854 26937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:38901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=33533
--tserver_master_addrs=127.13.195.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:46.966084 26937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:46.966360 26937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:46.972978 26973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.972978 26972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:46.973109 26975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:46.973802 26937 server_base.cc:1034] running on GCE node
I20260619 20:46:46.974006 26937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:46.974292 26937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:46.976032 26937 hybrid_clock.cc:648] HybridClock initialized: now 1781902006976012 us; error 33 us; skew 500 ppm
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:46.979218 26937 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:46:46.979745 26937 webserver.cc:460] Webserver started at http://127.13.195.65:33533/ using document root <none> and password file <none>
I20260619 20:46:46.980028 26937 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:46.980113 26937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:46:46 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:47.008111 26937 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260619 20:46:47.014926 26983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.015117 26937 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260619 20:46:47.015221 26937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "d3b92840a1374f9f8622514137b9428b"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "b23dbcdfb27553fcbf643866fad4a673"
server_key_iv: "9719852823d25402349142d303d189fe"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:47.015787 26937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:47.039183 26937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:47.041669 26937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:47.041833 26937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:47.042093 26937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:47.042488 26937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:47.042541 26937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.042579 26937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:47.042606 26937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.049569 26937 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:38901
I20260619 20:46:47.050052 26937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:46:47.050772 27096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:38901 every 8 connection(s)
I20260619 20:46:47.057265 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 26937
I20260619 20:46:47.057572 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:35469
--local_ip_for_outbound_sockets=127.13.195.66
--tserver_master_addrs=127.13.195.126:32779
--webserver_port=45849
--webserver_interface=127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902006, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:47.078814 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:47.051778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:51451 (local address 127.13.195.126:32779)
0619 20:46:47.052016 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:47.052028 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:47.054863 (+ 2835us) server_negotiation.cc:407] Connection header received
0619 20:46:47.055578 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:47.055587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:47.055663 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:47.055792 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:47.056701 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.058291 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.059961 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.060581 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.060666 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:47.074216 (+ 13550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:47.074223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:47.074223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:47.074238 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:47.076519 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.077135 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.077139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.077139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.077259 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.077563 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.077564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.077564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.077883 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:47.077955 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:47.078396 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 20:46:47.078569 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
I20260619 20:46:47.079761 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:47.051410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32779 (local address 127.13.195.65:51451)
0619 20:46:47.054518 (+ 3108us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:47.054579 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:47.055288 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:47.056014 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:47.056018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:47.056262 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:47.056543 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:47.056550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.058486 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.058487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:47.059841 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:47.059844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.060815 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.060816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:47.060953 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:47.061832 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:47.061844 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:47.074019 (+ 12175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:47.076635 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:47.076646 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:47.076648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:47.077017 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:47.077340 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:47.077341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:47.077341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:47.077483 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:47.078009 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:47.078015 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:47.078194 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 20:46:47.079342 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0619 20:46:47.079513 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3058,"thread_start_us":82,"threads_started":1}
I20260619 20:46:47.080729 27097 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32779
I20260619 20:46:47.080821 27097 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.081023 27097 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
W20260619 20:46:47.081908 27097 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:32779 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:46:47.081979 27097 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.082041 27097 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:47.082418 27097 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.082522 27097 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
W20260619 20:46:47.082837 27097 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:46:47.205070 27101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:47.205453 27101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:47.205523 27101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:47.207530 27101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:47.207605 27101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:47.207662 27101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:46:47.210114 27101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:35469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=45849
--tserver_master_addrs=127.13.195.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:47.210419 27101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:47.210788 27101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:47.216488 27107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:47.216491 27109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:47.216607 27106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:47.216898 27101 server_base.cc:1034] running on GCE node
I20260619 20:46:47.217125 27101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:47.217442 27101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:47.218621 27101 hybrid_clock.cc:648] HybridClock initialized: now 1781902007218627 us; error 44 us; skew 500 ppm
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:47.221344 27101 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:46:47.221837 27101 webserver.cc:460] Webserver started at http://127.13.195.66:45849/ using document root <none> and password file <none>
I20260619 20:46:47.222086 27101 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:47.222175 27101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:47.244073 27101 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:47.245401 27117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.245602 27101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:46:47.245702 27101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "39849a3169fe4b4dbc1c820e2597e4e7"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "bda1d28bfe8936eb02574abb0b960023"
server_key_iv: "b76e8728efb07a0647cea3d865c79be5"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:47.246148 27101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:47.271584 27101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:47.273869 27101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:47.274044 27101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:47.274323 27101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:47.274771 27101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:47.274839 27101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.274881 27101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:47.274901 27101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.282264 27101 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:35469
I20260619 20:46:47.282471 27230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:35469 every 8 connection(s)
I20260619 20:46:47.282779 27101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:46:47.289299 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27101
I20260619 20:46:47.289625 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.195.84:38235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:38257
--local_ip_for_outbound_sockets=127.13.195.67
--tserver_master_addrs=127.13.195.126:32779
--webserver_port=42391
--webserver_interface=127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:47.293512 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:47.283669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:47735 (local address 127.13.195.126:32779)
0619 20:46:47.283752 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:47.283754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:47.284018 (+ 264us) server_negotiation.cc:407] Connection header received
0619 20:46:47.284690 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:47.284692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:47.284702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:47.284743 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:47.285488 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.286098 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.287659 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.287859 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.287915 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:47.290510 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:47.290512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:47.290513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:47.290520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:47.292071 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.292523 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.292525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.292525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.292610 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.292858 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.292859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.292859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.292976 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:47.293051 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:47.293325 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 20:46:47.293420 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:46:47.293689 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:47.283638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32779 (local address 127.13.195.66:47735)
0619 20:46:47.283857 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:47.283907 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:47.284576 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:47.284834 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:47.284839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:47.285088 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:47.285395 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:47.285403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.286200 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.286201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:47.287564 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:47.287567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.287968 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.287969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:47.288108 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:47.288658 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:47.288669 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:47.290391 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:47.292204 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:47.292211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:47.292213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:47.292443 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:47.292684 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:47.292685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:47.292685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:47.292783 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:47.293042 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:47.293048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:47.293184 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 20:46:47.293292 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 20:46:47.293437 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260619 20:46:47.294194 27231 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32779
I20260619 20:46:47.294307 27231 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.294525 27231 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
W20260619 20:46:47.295187 27231 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:32779 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:46:47.295274 27231 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.295347 27231 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:47.295621 27231 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.295692 27231 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
W20260619 20:46:47.295956 27231 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:46:47.409482 27234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:47.409771 27234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:47.409807 27234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:47.411082 27234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:47.411190 27234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:47.411250 27234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:46:47.412774 27234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.195.84:38235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:38257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=42391
--tserver_master_addrs=127.13.195.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:47.413079 27234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:47.413334 27234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:47.418355 27241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:47.418351 27240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:47.418545 27243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:47.419130 27234 server_base.cc:1034] running on GCE node
I20260619 20:46:47.419345 27234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:47.419628 27234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:47.420789 27234 hybrid_clock.cc:648] HybridClock initialized: now 1781902007420776 us; error 29 us; skew 500 ppm
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:47.423408 27234 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:46:47.423906 27234 webserver.cc:460] Webserver started at http://127.13.195.67:42391/ using document root <none> and password file <none>
I20260619 20:46:47.424117 27234 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:47.424168 27234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:47.448274 27234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:47.449338 27251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.449544 27234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:46:47.449640 27234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "17b24e36dfd7446a9cd25339ccf5a96f"
format_stamp: "Formatted at 2026-06-19 20:46:46 on dist-test-slave-n2td"
server_key: "2b3012d03b513eeaa5f21de695c685b9"
server_key_iv: "00c72761d847fa8c898f9c5795d4da1c"
server_key_version: "kuduclusterkey@0"
I20260619 20:46:47.450100 27234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:47.472234 27234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:47.474241 27234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:47.474501 27234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:47.474819 27234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:47.475195 27234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:47.475256 27234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.475297 27234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:47.475324 27234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:47.482790 27234 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:38257
I20260619 20:46:47.482839 27364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:38257 every 8 connection(s)
I20260619 20:46:47.483292 27234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:47.491199 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27234
I20260619 20:46:47.493211 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:47.484426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:39531 (local address 127.13.195.126:32779)
0619 20:46:47.484474 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:47.484476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:47.484486 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:46:47.485029 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:47.485031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:47.485044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:47.485108 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:47.485751 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.486253 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.487352 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.487566 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.487658 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:47.490026 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:47.490027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:47.490027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:47.490036 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:47.491717 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.492214 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.492215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.492215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.492279 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.492545 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.492545 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.492546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.492674 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:47.492752 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:47.493035 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 20:46:47.493116 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:46:47.493432 27367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:47.484137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:32779 (local address 127.13.195.67:39531)
0619 20:46:47.484357 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:47.484400 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:47.484897 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:47.485184 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:47.485188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:47.485380 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:47.485647 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:47.485653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.486359 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.486360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:47.487253 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:47.487255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.487689 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.487691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:47.487819 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:47.488424 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:47.488436 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:47.489944 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:47.491814 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:47.491822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:47.491824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:47.492140 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:47.492351 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:47.492351 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:47.492352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:47.492464 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:47.492743 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:47.492749 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:47.492898 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 20:46:47.493014 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 20:46:47.493176 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260619 20:46:47.494109 27365 heartbeater.cc:347] Connected to a master server at 127.13.195.126:32779
I20260619 20:46:47.494221 27365 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.494405 27365 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
W20260619 20:46:47.495074 27365 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:32779 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:46:47.495159 27365 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.495229 27365 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:47.495597 27365 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:47.495687 27365 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
W20260619 20:46:47.496002 27365 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:46:47.505810 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:47.492198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.13.195.126:32779)
0619 20:46:47.492326 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:47.492328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:47.493904 (+ 1576us) server_negotiation.cc:407] Connection header received
0619 20:46:47.494024 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:47.494025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:47.494036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:47.494102 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:47.495352 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.496111 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.497171 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:47.497363 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:47.497418 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:47.502715 (+ 5297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:47.502716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:47.502717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:47.502724 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:47.504463 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.504937 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.504938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.504939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.505008 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:47.505269 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:47.505270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:47.505270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:47.505418 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:47.505548 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:47.505669 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 20:46:47.505738 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
Jun 19 20:46:47 dist-test-slave-n2td krb5kdc[25903](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:48.083014 27097 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:48.083184 27097 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:48.083521 26896 ts_manager.cc:194] Registered new tserver with Master: d3b92840a1374f9f8622514137b9428b (127.13.195.65:38901)
I20260619 20:46:48.084539 26896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:51451
I20260619 20:46:48.296123 27231 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:48.296301 27231 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:48.296602 26896 ts_manager.cc:194] Registered new tserver with Master: 39849a3169fe4b4dbc1c820e2597e4e7 (127.13.195.66:35469)
I20260619 20:46:48.297011 26896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:47735
Jun 19 20:46:48 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:48.496204 27365 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:48.496358 27365 heartbeater.cc:511] Master 127.13.195.126:32779 requested a full tablet report, sending...
I20260619 20:46:48.496697 26896 ts_manager.cc:194] Registered new tserver with Master: 17b24e36dfd7446a9cd25339ccf5a96f (127.13.195.67:38257)
I20260619 20:46:48.497231 26896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:39531
I20260619 20:46:48.507786 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:46:48.521343 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.508827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.13.195.126:32779)
0619 20:46:48.509076 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:48.509078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:48.509085 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:46:48.509107 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:48.509108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:48.509120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:48.509209 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:48.510011 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.510842 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.511967 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.512276 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.512363 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.518097 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:48.518099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:48.518099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:48.518108 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:48.520060 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:48.520447 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:48.520448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:48.520448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:48.520513 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:48.520767 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:48.520768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:48.520768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:48.520915 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:48.521022 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:48.521144 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 20:46:48.521232 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
I20260619 20:46:48.523813 26896 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:46:48.526156 26896 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:46:48.535120 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.529571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38901 (local address 127.0.0.1:40012)
0619 20:46:48.530457 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:48.530503 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:48.530594 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:48.531065 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:48.531070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:48.531073 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:48.531167 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.531170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.532555 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.532556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.533878 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.533880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.534848 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.534849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.534923 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.534940 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 20:46:48.534984 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:46:48.535032 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":97,"threads_started":1}
I20260619 20:46:48.535621 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.529575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:35469 (local address 127.0.0.1:40662)
0619 20:46:48.530722 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:48.530731 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:48.530769 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:48.531329 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:48.531330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:48.531332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:48.531386 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.531388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.533021 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.533022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.534111 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.534114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.535408 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.535409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.535472 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.535477 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:46:48.535508 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 20:46:48.535548 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"thread_start_us":103,"threads_started":1}
I20260619 20:46:48.536201 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.530026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.13.195.66:35469)
0619 20:46:48.531172 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:48.531176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:48.531185 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:46:48.531216 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:48.531218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:48.531232 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:48.531455 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:48.531523 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.532918 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.534213 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.535303 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.535973 (+ 670us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.536044 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:48.536072 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:46:48.536113 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1113,"thread_start_us":80,"threads_started":1}
I20260619 20:46:48.537026 27165 tablet_service.cc:1467] Processing CreateTablet for tablet a0ee52b447a641eea53eb554e4ef22c7 (DEFAULT_TABLE table=test-table [id=40f91c5b07c440fc9bddb824dc4e2571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:48.537431 27165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ee52b447a641eea53eb554e4ef22c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:48.539117 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.529822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38257 (local address 127.0.0.1:60426)
0619 20:46:48.535528 (+ 5706us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:48.535538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:48.535573 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:48.535794 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:48.535794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:48.535796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:48.535824 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.535826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.536996 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.536997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.537991 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.537993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.538941 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.538942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.538995 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.538998 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:46:48.539024 (+ 26us) client_negotiation.cc:240] Negotiation successful
0619 20:46:48.539058 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5694,"thread_start_us":65,"threads_started":1}
I20260619 20:46:48.539404 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.530145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60426 (local address 127.13.195.67:38257)
0619 20:46:48.530307 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:48.530313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:48.535646 (+ 5333us) server_negotiation.cc:407] Connection header received
0619 20:46:48.535691 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:48.535694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:48.535711 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:48.535882 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:48.535953 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.536905 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.538088 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.538844 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.539208 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.539274 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:48.539299 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 20:46:48.539335 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260619 20:46:48.539733 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.529844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.13.195.65:38901)
0619 20:46:48.530873 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:48.530879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:48.530891 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:46:48.530928 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:48.530930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:48.530947 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:48.531026 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:48.531277 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.532452 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.533992 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.534754 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.539504 (+ 4750us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.539575 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:48.539600 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 20:46:48.539659 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":56,"threads_started":1}
I20260619 20:46:48.540269 27031 tablet_service.cc:1467] Processing CreateTablet for tablet a0ee52b447a641eea53eb554e4ef22c7 (DEFAULT_TABLE table=test-table [id=40f91c5b07c440fc9bddb824dc4e2571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:48.540508 27031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ee52b447a641eea53eb554e4ef22c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:48.541133 27398 tablet_bootstrap.cc:492] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Bootstrap starting.
I20260619 20:46:48.541774 27398 tablet_bootstrap.cc:654] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:48.541967 27298 tablet_service.cc:1467] Processing CreateTablet for tablet a0ee52b447a641eea53eb554e4ef22c7 (DEFAULT_TABLE table=test-table [id=40f91c5b07c440fc9bddb824dc4e2571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:48.542071 27398 log.cc:824] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:48.542307 27298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ee52b447a641eea53eb554e4ef22c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:48.543979 27400 tablet_bootstrap.cc:492] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b: Bootstrap starting.
I20260619 20:46:48.544636 27400 tablet_bootstrap.cc:654] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:48.544970 27400 log.cc:824] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:48.547049 27398 tablet_bootstrap.cc:492] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: No bootstrap required, opened a new log
I20260619 20:46:48.547176 27398 ts_tablet_manager.cc:1397] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 20:46:48.547504 27402 tablet_bootstrap.cc:492] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Bootstrap starting.
I20260619 20:46:48.548216 27402 tablet_bootstrap.cc:654] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:48.548790 27402 log.cc:824] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:48.550129 27400 tablet_bootstrap.cc:492] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b: No bootstrap required, opened a new log
I20260619 20:46:48.550252 27400 ts_tablet_manager.cc:1397] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 20:46:48.550321 27402 tablet_bootstrap.cc:492] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: No bootstrap required, opened a new log
I20260619 20:46:48.550462 27402 ts_tablet_manager.cc:1397] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:46:48.552644 27400 raft_consensus.cc:348] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.552922 27400 raft_consensus.cc:374] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:48.553009 27400 raft_consensus.cc:729] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b92840a1374f9f8622514137b9428b, State: Initialized, Role: FOLLOWER
I20260619 20:46:48.553030 27402 raft_consensus.cc:348] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.553238 27402 raft_consensus.cc:374] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:48.553267 27400 consensus_queue.cc:260] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.553329 27402 raft_consensus.cc:729] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17b24e36dfd7446a9cd25339ccf5a96f, State: Initialized, Role: FOLLOWER
I20260619 20:46:48.553471 27402 consensus_queue.cc:260] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.553642 27400 ts_tablet_manager.cc:1428] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:46:48.553791 27402 ts_tablet_manager.cc:1428] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:46:48.553845 27097 heartbeater.cc:503] Master 127.13.195.126:32779 was elected leader, sending a full tablet report...
I20260619 20:46:48.553942 27365 heartbeater.cc:503] Master 127.13.195.126:32779 was elected leader, sending a full tablet report...
I20260619 20:46:48.549674 27398 raft_consensus.cc:348] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.554965 27398 raft_consensus.cc:374] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:48.555034 27398 raft_consensus.cc:729] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39849a3169fe4b4dbc1c820e2597e4e7, State: Initialized, Role: FOLLOWER
I20260619 20:46:48.555164 27398 consensus_queue.cc:260] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.555415 27398 ts_tablet_manager.cc:1428] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260619 20:46:48.555445 27231 heartbeater.cc:503] Master 127.13.195.126:32779 was elected leader, sending a full tablet report...
W20260619 20:46:48.557309 27098 tablet.cc:2369] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 20:46:48 dist-test-slave-n2td krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902007, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for HTTP/127.13.195.84@KRBTEST.COM
I20260619 20:46:48.628046 27405 raft_consensus.cc:482] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:46:48.628232 27405 raft_consensus.cc:504] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.628666 27405 leader_election.cc:288] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3b92840a1374f9f8622514137b9428b (127.13.195.65:38901), 39849a3169fe4b4dbc1c820e2597e4e7 (127.13.195.66:35469)
I20260619 20:46:48.633621 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.629006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:53189 (local address 127.13.195.66:35469)
0619 20:46:48.629073 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:48.629075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:48.630315 (+ 1240us) server_negotiation.cc:407] Connection header received
0619 20:46:48.630412 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:48.630413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:48.630423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:48.630478 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:48.630697 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.631348 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.632662 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.633220 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.633272 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.633281 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:48.633504 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 20:46:48.633548 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:46:48.634267 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.628816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:35469 (local address 127.13.195.67:53189)
0619 20:46:48.630261 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:48.630270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:48.630340 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:48.630548 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:48.630549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:48.630550 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:48.630618 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.630619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.631431 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.631432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.632553 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.632557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.633353 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.633354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.633425 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.633429 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:46:48.633943 (+ 514us) client_negotiation.cc:240] Negotiation successful
0619 20:46:48.633990 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1430,"thread_start_us":90,"threads_started":1}
I20260619 20:46:48.634814 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ee52b447a641eea53eb554e4ef22c7" candidate_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" is_pre_election: true
I20260619 20:46:48.635001 27185 raft_consensus.cc:2454] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17b24e36dfd7446a9cd25339ccf5a96f in term 0.
I20260619 20:46:48.635264 27253 leader_election.cc:302] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17b24e36dfd7446a9cd25339ccf5a96f, 39849a3169fe4b4dbc1c820e2597e4e7; no voters:
I20260619 20:46:48.635404 27405 raft_consensus.cc:2781] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:46:48.635453 27405 raft_consensus.cc:482] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:46:48.635473 27405 raft_consensus.cc:3037] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:48.636446 27405 raft_consensus.cc:504] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.636587 27405 leader_election.cc:288] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [CANDIDATE]: Term 1 election: Requested vote from peers d3b92840a1374f9f8622514137b9428b (127.13.195.65:38901), 39849a3169fe4b4dbc1c820e2597e4e7 (127.13.195.66:35469)
I20260619 20:46:48.636922 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ee52b447a641eea53eb554e4ef22c7" candidate_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39849a3169fe4b4dbc1c820e2597e4e7"
I20260619 20:46:48.637001 27185 raft_consensus.cc:3037] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:48.638320 27185 raft_consensus.cc:2454] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17b24e36dfd7446a9cd25339ccf5a96f in term 1.
I20260619 20:46:48.638594 27253 leader_election.cc:302] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17b24e36dfd7446a9cd25339ccf5a96f, 39849a3169fe4b4dbc1c820e2597e4e7; no voters:
I20260619 20:46:48.638805 27405 raft_consensus.cc:2781] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:48.638965 27405 raft_consensus.cc:686] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 1 LEADER]: Becoming Leader. State: Replica: 17b24e36dfd7446a9cd25339ccf5a96f, State: Running, Role: LEADER
I20260619 20:46:48.639057 27405 consensus_queue.cc:237] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } }
I20260619 20:46:48.639882 26894 catalog_manager.cc:5495] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f reported cstate change: term changed from 0 to 1, leader changed from <none> to 17b24e36dfd7446a9cd25339ccf5a96f (127.13.195.67). New cstate: current_term: 1 leader_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38257 } health_report { overall_health: HEALTHY } } }
I20260619 20:46:48.641744 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.629013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:32987 (local address 127.13.195.65:38901)
0619 20:46:48.629077 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:48.629079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:48.630188 (+ 1109us) server_negotiation.cc:407] Connection header received
0619 20:46:48.630262 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:48.630264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:48.630276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:48.630334 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:48.630613 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.631362 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.640804 (+ 9442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.641356 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.641403 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.641412 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:48.641631 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 20:46:48.641677 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:46:48.641976 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.628802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38901 (local address 127.13.195.67:32987)
0619 20:46:48.630094 (+ 1292us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:48.630107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:48.630174 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:48.630397 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:48.630398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:48.630401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:48.630511 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.630513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.637329 (+ 6816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.637330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.640691 (+ 3361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:48.640694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.641469 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.641470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:48.641538 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.641541 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:46:48.641861 (+ 320us) client_negotiation.cc:240] Negotiation successful
0619 20:46:48.641905 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"thread_start_us":103,"threads_started":1}
I20260619 20:46:48.642295 27051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ee52b447a641eea53eb554e4ef22c7" candidate_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b92840a1374f9f8622514137b9428b" is_pre_election: true
I20260619 20:46:48.642486 27050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ee52b447a641eea53eb554e4ef22c7" candidate_uuid: "17b24e36dfd7446a9cd25339ccf5a96f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b92840a1374f9f8622514137b9428b"
I20260619 20:46:48.642885 27051 raft_consensus.cc:2454] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17b24e36dfd7446a9cd25339ccf5a96f in term 0.
I20260619 20:46:48.656032 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.650795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.13.195.67:38257)
0619 20:46:48.650970 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:48.650973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:48.650980 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:46:48.651019 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:48.651020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:48.651042 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:48.651130 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:46:48.651311 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.652085 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.654833 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:48.655127 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:48.655222 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:48.655473 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:46:48.655616 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:46:48.655691 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:48.655925 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 20:46:48.655948 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260619 20:46:48.659806 27185 raft_consensus.cc:1261] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 1 FOLLOWER]: Refusing update from remote peer 17b24e36dfd7446a9cd25339ccf5a96f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:48.660212 27405 consensus_queue.cc:1035] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [LEADER]: Connected to new peer: Peer: permanent_uuid: "39849a3169fe4b4dbc1c820e2597e4e7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:48.662479 27051 raft_consensus.cc:3037] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:48.663582 27051 raft_consensus.cc:1261] T a0ee52b447a641eea53eb554e4ef22c7 P d3b92840a1374f9f8622514137b9428b [term 1 FOLLOWER]: Refusing update from remote peer 17b24e36dfd7446a9cd25339ccf5a96f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:48.663904 27410 consensus_queue.cc:1035] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3b92840a1374f9f8622514137b9428b" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:48.664359 27414 mvcc.cc:205] Tried to move back new op lower bound from 7298670627467370496 to 7298670627386142720. Current Snapshot: MvccSnapshot[applied={T|T < 7298670627467370496}]
I20260619 20:46:48.667733 26894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:48.667820 26894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:48.668645 26894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 369212b714dc43dfb18517bf7c1899d8: Sending DeleteTablet for 3 replicas of tablet a0ee52b447a641eea53eb554e4ef22c7
I20260619 20:46:48.669085 27031 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ee52b447a641eea53eb554e4ef22c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:48 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:40012
I20260619 20:46:48.669406 27165 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ee52b447a641eea53eb554e4ef22c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:48 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:40662
I20260619 20:46:48.669919 27298 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ee52b447a641eea53eb554e4ef22c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:48 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:60426
I20260619 20:46:48.671155 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 26937
I20260619 20:46:48.674656 27413 mvcc.cc:205] Tried to move back new op lower bound from 7298670627467370496 to 7298670627386142720. Current Snapshot: MvccSnapshot[applied={T|T < 7298670627467370496 or (T in {7298670627467370496})}]
I20260619 20:46:48.675076 27425 tablet_replica.cc:330] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: stopping tablet replica
I20260619 20:46:48.675468 27425 raft_consensus.cc:2229] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:46:48.675618 27425 raft_consensus.cc:2258] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:48.679049 27425 ts_tablet_manager.cc:1905] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:48.680863 27425 ts_tablet_manager.cc:1918] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:48.680997 27425 log.cc:1196] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/a0ee52b447a641eea53eb554e4ef22c7
I20260619 20:46:48.681367 27425 ts_tablet_manager.cc:1939] T a0ee52b447a641eea53eb554e4ef22c7 P 39849a3169fe4b4dbc1c820e2597e4e7: Deleting consensus metadata
I20260619 20:46:48.681897 26884 catalog_manager.cc:4841] TS 39849a3169fe4b4dbc1c820e2597e4e7 (127.13.195.66:35469): tablet a0ee52b447a641eea53eb554e4ef22c7 (table test-table [id=40f91c5b07c440fc9bddb824dc4e2571]) successfully deleted
I20260619 20:46:48.682529 27426 tablet_replica.cc:330] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: stopping tablet replica
I20260619 20:46:48.682675 27426 raft_consensus.cc:2229] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:46:48.682843 27426 raft_consensus.cc:2258] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:48.683521 27426 ts_tablet_manager.cc:1905] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 20:46:48.683736 26885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:38901 (error 108)
I20260619 20:46:48.684576 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27101
I20260619 20:46:48.684592 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:48.684435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38901 (local address 127.0.0.1:40014)
0619 20:46:48.684487 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:48.684554 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:46:48.684715 26885 catalog_manager.cc:4584] TS d3b92840a1374f9f8622514137b9428b (127.13.195.65:38901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0ee52b447a641eea53eb554e4ef22c7: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38901: connect: Connection refused (error 111)
I20260619 20:46:48.685425 27426 ts_tablet_manager.cc:1918] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:48.685491 27426 log.cc:1196] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/a0ee52b447a641eea53eb554e4ef22c7
I20260619 20:46:48.685786 27426 ts_tablet_manager.cc:1939] T a0ee52b447a641eea53eb554e4ef22c7 P 17b24e36dfd7446a9cd25339ccf5a96f: Deleting consensus metadata
W20260619 20:46:48.683522 27254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:38901 (error 108)
I20260619 20:46:48.692662 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27234
I20260619 20:46:48.698797 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 26866
I20260619 20:46:48.708142 14093 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 20:46:49.052917 14093 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 20:46:49.053007 14093 mini_ranger.cc:66] Stopping Ranger...
I20260619 20:46:49.397908 14093 mini_ranger.cc:68] Stopped Ranger
2026-06-19 20:46:49.398 UTC [25933] LOG: received smart shutdown request
2026-06-19 20:46:49.402 UTC [25933] LOG: background worker "logical replication launcher" (PID 25942) exited with exit code 1
2026-06-19 20:46:49.402 UTC [25937] LOG: shutting down
2026-06-19 20:46:49.567 UTC [25933] LOG: database system is shut down
2026-06-19T20:46:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47047 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:46:49 dist-test-slave-n2td krb5kdc[27437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:46:49 dist-test-slave-n2td krb5kdc[27437](info): set up 2 sockets
Jun 19 20:46:49 dist-test-slave-n2td krb5kdc[27437](info): commencing operation
krb5kdc: starting...
W20260619 20:46:50.713383 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:46:50 dist-test-slave-n2td krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:46:50.833331 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45243
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45243,127.13.195.125:45755,127.13.195.124:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:50.915272 27450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:50.915488 27450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:50.915512 27450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:50.916723 27450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:50.916772 27450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:50.916787 27450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:50.916800 27450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:50.916854 27450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:50.918360 27450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45243,127.13.195.125:45755,127.13.195.124:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:50.918650 27450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:50.918900 27450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:50.923972 27456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:50.924000 27450 server_base.cc:1034] running on GCE node
W20260619 20:46:50.924007 27458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:50.923972 27455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:50.924443 27450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:50.924737 27450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:50.925926 27450 hybrid_clock.cc:648] HybridClock initialized: now 1781902010925919 us; error 38 us; skew 500 ppm
Jun 19 20:46:50 dist-test-slave-n2td krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:50.928511 27450 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:46:50.929018 27450 webserver.cc:460] Webserver started at http://127.13.195.126:46553/ using document root <none> and password file <none>
I20260619 20:46:50.929275 27450 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:50.929344 27450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:50.929497 27450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:50.930718 27450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "b8411d09d0334ba1955e53ba565f1930"
format_stamp: "Formatted at 2026-06-19 20:46:50 on dist-test-slave-n2td"
server_key: "5650627fb727c411e67e009b4613b9b9"
server_key_iv: "07eda47e5f9036c81c0531f0159a970c"
server_key_version: "encryptionkey@0"
I20260619 20:46:50.931104 27450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "b8411d09d0334ba1955e53ba565f1930"
format_stamp: "Formatted at 2026-06-19 20:46:50 on dist-test-slave-n2td"
server_key: "5650627fb727c411e67e009b4613b9b9"
server_key_iv: "07eda47e5f9036c81c0531f0159a970c"
server_key_version: "encryptionkey@0"
I20260619 20:46:50.932538 27450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:50.933333 27465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:50.933531 27450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:50.933689 27450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "b8411d09d0334ba1955e53ba565f1930"
format_stamp: "Formatted at 2026-06-19 20:46:50 on dist-test-slave-n2td"
server_key: "5650627fb727c411e67e009b4613b9b9"
server_key_iv: "07eda47e5f9036c81c0531f0159a970c"
server_key_version: "encryptionkey@0"
I20260619 20:46:50.933768 27450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:50.968515 27450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:50.971076 27450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:50.971263 27450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:50.975601 27450 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:45243
I20260619 20:46:50.975677 27517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:45243 every 8 connection(s)
I20260619 20:46:50.976061 27450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:46:50.976933 27518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:50.978642 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27450
I20260619 20:46:50.978751 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:46:50.978572 27518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:50.978948 14093 external_mini_cluster.cc:1428] Setting key 7c7a48559d0dee3bcc542ab16c399393
Jun 19 20:46:50 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.125@KRBTEST.COM" created.
I20260619 20:46:50.989447 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:50.979399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45243 (local address 127.0.0.1:51104)
0619 20:46:50.979620 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:50.979671 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:50.980193 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:50.980543 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:50.980546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:50.980915 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:50.981159 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:50.981167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:50.982185 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:50.982186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:50.982929 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:50.982932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:50.983446 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:50.983447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:50.983510 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:50.984078 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:50.984086 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:50.985677 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:50.987698 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:50.987704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:50.987706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:50.987917 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:50.988211 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:50.988212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:50.988213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:50.988316 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:50.988785 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:50.988790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:50.988855 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 20:46:50.989013 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 20:46:50.989269 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":96,"thread_start_us":88,"threads_started":1}
I20260619 20:46:50.989451 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:50.979605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.13.195.126:45243)
0619 20:46:50.979753 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:50.979758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:50.979797 (+ 39us) server_negotiation.cc:407] Connection header received
0619 20:46:50.980378 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:50.980380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:50.980408 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:50.980496 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:50.981264 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:50.982062 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:50.983029 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:50.983346 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:50.983474 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:50.985776 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:50.985783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:50.985783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:50.985797 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:50.987585 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:50.988028 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:50.988030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:50.988031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:50.988120 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:50.988403 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:50.988404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:50.988404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:50.988698 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:50.988802 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:50.989042 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 20:46:50.989188 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260619 20:46:50.991345 27518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:50.991739 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:50.991579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35128)
0619 20:46:50.991664 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:50.991714 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 20:46:50.991928 27469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
I20260619 20:46:50.992408 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:50.992273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35138)
0619 20:46:50.992355 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:50.992387 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 20:46:50.992602 27518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:45755: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab.
Entry for principal kudu/127.13.195.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab.
I20260619 20:46:51.003727 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:45755
--webserver_interface=127.13.195.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab
--principal=kudu/127.13.195.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45243,127.13.195.125:45755,127.13.195.124:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:46:51.041828 27518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } attempt: 1
I20260619 20:46:51.042290 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.042158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35140)
0619 20:46:51.042220 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.042261 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:46:51.042641 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.042544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35144)
0619 20:46:51.042589 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.042621 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:46:51.042732 27518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:45755: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
W20260619 20:46:51.087217 27525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:51.087422 27525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:51.087445 27525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:51.088605 27525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:51.088654 27525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:51.088668 27525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:51.088681 27525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:51.088735 27525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:51.090234 27525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45243,127.13.195.125:45755,127.13.195.124:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab
--principal=kudu/127.13.195.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:45755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.195.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:51.090538 27525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:51.090788 27525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:46:51.095744 27525 server_base.cc:1034] running on GCE node
W20260619 20:46:51.095749 27533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:51.095778 27531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:51.095858 27530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:51.096357 27525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:51.096730 27525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:51.097905 27525 hybrid_clock.cc:648] HybridClock initialized: now 1781902011097886 us; error 50 us; skew 500 ppm
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:51.100559 27525 init.cc:377] Logged in from keytab as kudu/127.13.195.125@KRBTEST.COM (short username kudu)
I20260619 20:46:51.100993 27525 webserver.cc:460] Webserver started at http://127.13.195.125:40227/ using document root <none> and password file <none>
I20260619 20:46:51.101234 27525 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:51.101279 27525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:51.101470 27525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:51.102614 27525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data/instance:
uuid: "144b8d8ffd674de0804a0ec13a39c349"
format_stamp: "Formatted at 2026-06-19 20:46:51 on dist-test-slave-n2td"
server_key: "9cefb2686ce3bf561ddb575b2e705e6f"
server_key_iv: "3044bfd402da241d68485ecd8da0685e"
server_key_version: "encryptionkey@0"
I20260619 20:46:51.102973 27525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/wal/instance:
uuid: "144b8d8ffd674de0804a0ec13a39c349"
format_stamp: "Formatted at 2026-06-19 20:46:51 on dist-test-slave-n2td"
server_key: "9cefb2686ce3bf561ddb575b2e705e6f"
server_key_iv: "3044bfd402da241d68485ecd8da0685e"
server_key_version: "encryptionkey@0"
I20260619 20:46:51.104278 27525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:51.105010 27540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:51.105185 27525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:51.105248 27525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/wal
uuid: "144b8d8ffd674de0804a0ec13a39c349"
format_stamp: "Formatted at 2026-06-19 20:46:51 on dist-test-slave-n2td"
server_key: "9cefb2686ce3bf561ddb575b2e705e6f"
server_key_iv: "3044bfd402da241d68485ecd8da0685e"
server_key_version: "encryptionkey@0"
I20260619 20:46:51.105336 27525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:51.110863 27518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } attempt: 2
I20260619 20:46:51.111279 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.111155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35150)
0619 20:46:51.111213 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.111252 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:46:51.111658 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.111576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35152)
0619 20:46:51.111619 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.111643 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:46:51.111763 27518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:45755: Network error: Client connection negotiation failed: client connection to 127.13.195.125:45755: connect: Connection refused (error 111)
I20260619 20:46:51.124692 27525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:51.127159 27525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:51.127334 27525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:51.131776 27525 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.125:45755
I20260619 20:46:51.131808 27592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.125:45755 every 8 connection(s)
I20260619 20:46:51.132202 27525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
I20260619 20:46:51.133080 27593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:51.134758 27593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:51.139551 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27525
I20260619 20:46:51.139674 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-1/wal/instance
I20260619 20:46:51.139882 14093 external_mini_cluster.cc:1428] Setting key b6c5984246c9957c37f17d71045a7445
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:51.144712 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.135571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.13.195.126:45243)
0619 20:46:51.135638 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.135640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.135834 (+ 194us) server_negotiation.cc:407] Connection header received
0619 20:46:51.136291 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.136292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.136304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.136366 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.137301 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.137895 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.138969 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.139173 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.139230 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.141561 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.141563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.141563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.141572 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.143059 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.143637 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.143638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.143639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.143719 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.144042 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.144042 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.144043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.144151 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.144204 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.144555 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.144628 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:46:51.145022 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.135532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45243 (local address 127.0.0.1:51106)
0619 20:46:51.135700 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.135744 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.136212 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.136419 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.136422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.136949 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.137198 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.137204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.137997 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.137998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.138893 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.138895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.139273 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.139275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.139393 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.140043 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.140055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.141463 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.143146 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.143155 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.143157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.143506 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.143800 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.143801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.143802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.143962 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.144221 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.144228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.144405 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.144539 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.144740 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260619 20:46:51.145653 27593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.195.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.124@KRBTEST.COM" created.
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:46:51.154633 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.146014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.13.195.125:45755)
0619 20:46:51.146207 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.146211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.146226 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:46:51.146242 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.146244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.146254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.146311 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.146686 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.147369 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.148414 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.148708 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.148771 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.151791 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.151801 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.151801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.151809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.153410 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.153809 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.153811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.153812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.153862 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.154065 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.154066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.154066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.154195 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.154257 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.154406 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.154551 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260619 20:46:51.154794 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.145935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35164)
0619 20:46:51.146002 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.146012 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.146052 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.146347 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.146348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.146536 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.146602 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.146604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.147463 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.147464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.148337 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.148340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.148803 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.148804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.148871 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.149335 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.149340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.151697 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.153497 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.153498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.153498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.153728 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.153933 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.153934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.153934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.154005 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.154259 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.154260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.154307 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.154384 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.154645 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":122}
I20260619 20:46:51.155428 27593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:51.155815 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.155671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50610)
0619 20:46:51.155725 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.155780 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 20:46:51.155941 27544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
I20260619 20:46:51.156375 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.156280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50612)
0619 20:46:51.156332 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.156357 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 20:46:51.156503 27593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37471: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab.
Entry for principal kudu/127.13.195.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab.
I20260619 20:46:51.166584 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:37471
--webserver_interface=127.13.195.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab
--principal=kudu/127.13.195.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45243,127.13.195.125:45755,127.13.195.124:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:46:51.202911 27518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } attempt: 3
I20260619 20:46:51.205765 27593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } attempt: 1
I20260619 20:46:51.206204 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.206055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50626)
0619 20:46:51.206119 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.206172 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 20:46:51.206782 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.206676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50634)
0619 20:46:51.206727 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.206756 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:46:51.206867 27593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37471: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:46:51.210770 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.203354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35170)
0619 20:46:51.203418 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.203430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.203474 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.203634 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.203635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.203820 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.203898 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.203901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.204553 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.204554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.205356 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.205359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.205844 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.205846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.205953 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.206509 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.206515 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.208067 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.209678 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.209680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.209680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.209855 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.210110 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.210111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.210111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.210168 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.210489 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.210490 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.210546 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.210600 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.210701 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 20:46:51.210829 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.203430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.13.195.125:45755)
0619 20:46:51.203495 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.203497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.203507 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:46:51.203532 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.203533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.203542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.203609 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.203973 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.204474 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.205483 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.205734 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.205847 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.208164 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.208165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.208165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.208172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.209568 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.209960 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.209961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.209962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.210033 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.210236 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.210238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.210238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.210394 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.210499 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.210648 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.210741 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:46:51.211266 27518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:51.211601 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.211499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50644)
0619 20:46:51.211548 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.211579 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:46:51.211951 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.211893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50648)
0619 20:46:51.211917 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.211938 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 20:46:51.212059 27518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37471: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
I20260619 20:46:51.243218 27518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } attempt: 1
I20260619 20:46:51.243666 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.243533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50656)
0619 20:46:51.243599 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.243638 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 20:46:51.244028 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.243930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50660)
0619 20:46:51.243976 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.244003 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:46:51.244118 27518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37471: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
W20260619 20:46:51.250649 27600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:51.250842 27600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:51.250865 27600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:51.252038 27600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:51.252077 27600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:51.252092 27600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:51.252136 27600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:51.252163 27600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:51.253691 27600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45243,127.13.195.125:45755,127.13.195.124:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab
--principal=kudu/127.13.195.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.195.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:51.253940 27600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:51.254195 27600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:51.259102 27606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:51.259200 27600 server_base.cc:1034] running on GCE node
W20260619 20:46:51.259109 27605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:51.259109 27608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:51.259630 27600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:51.259887 27600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:51.261029 27600 hybrid_clock.cc:648] HybridClock initialized: now 1781902011261010 us; error 36 us; skew 500 ppm
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:51.263499 27600 init.cc:377] Logged in from keytab as kudu/127.13.195.124@KRBTEST.COM (short username kudu)
I20260619 20:46:51.263947 27600 webserver.cc:460] Webserver started at http://127.13.195.124:45261/ using document root <none> and password file <none>
I20260619 20:46:51.264144 27600 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:51.264184 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:51.264272 27600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:51.265293 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data/instance:
uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb"
format_stamp: "Formatted at 2026-06-19 20:46:51 on dist-test-slave-n2td"
server_key: "02449c45caf69997f5d88df0444b74a3"
server_key_iv: "8b4293a4e2db95ee63183d1a377fe2a3"
server_key_version: "encryptionkey@0"
I20260619 20:46:51.265594 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/wal/instance:
uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb"
format_stamp: "Formatted at 2026-06-19 20:46:51 on dist-test-slave-n2td"
server_key: "02449c45caf69997f5d88df0444b74a3"
server_key_iv: "8b4293a4e2db95ee63183d1a377fe2a3"
server_key_version: "encryptionkey@0"
I20260619 20:46:51.266893 27600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:46:51.267560 27615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:51.267751 27600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:46:51.267832 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/wal
uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb"
format_stamp: "Formatted at 2026-06-19 20:46:51 on dist-test-slave-n2td"
server_key: "02449c45caf69997f5d88df0444b74a3"
server_key_iv: "8b4293a4e2db95ee63183d1a377fe2a3"
server_key_version: "encryptionkey@0"
I20260619 20:46:51.267884 27600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:51.274997 27593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } attempt: 2
I20260619 20:46:51.275408 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.275284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50670)
0619 20:46:51.275340 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.275377 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:46:51.275738 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.275661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50682)
0619 20:46:51.275701 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.275722 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 20:46:51.275848 27593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37471: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37471: connect: Connection refused (error 111)
I20260619 20:46:51.289850 27600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:51.292269 27600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:51.292459 27600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:51.296600 27600 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.124:37471
I20260619 20:46:51.296656 27667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.124:37471 every 8 connection(s)
I20260619 20:46:51.297015 27600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
I20260619 20:46:51.297910 27668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:51.299441 27668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:51.301254 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27600
I20260619 20:46:51.301383 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901857247738-14093-0/minicluster-data/master-2/wal/instance
I20260619 20:46:51.301587 14093 external_mini_cluster.cc:1428] Setting key 286eb66fe0dcb3bddff2a7da6e615e89
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:46:51.309445 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.300128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.13.195.126:45243)
0619 20:46:51.300195 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.300197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.300378 (+ 181us) server_negotiation.cc:407] Connection header received
0619 20:46:51.300824 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.300826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.300834 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.300893 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.301581 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.302238 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.303418 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.303603 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.303656 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.306117 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.306118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.306118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.306134 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.307808 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.308339 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.308341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.308341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.308412 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.308723 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.308724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.308724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.308860 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.308925 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.309265 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.309344 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:46:51.309513 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.300082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45243 (local address 127.0.0.1:51108)
0619 20:46:51.300241 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.300284 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.300744 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.300961 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.300965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.301219 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.301473 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.301479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.302322 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.302323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.303335 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.303338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.303679 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.303681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.303826 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.304377 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.304385 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.306020 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.307877 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.307884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.307886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.308226 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.308492 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.308493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.308493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.308629 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.308956 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.308962 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.309137 (+ 175us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.309221 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.309330 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260619 20:46:51.310400 27668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:51.310998 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.301970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.13.195.124:37471)
0619 20:46:51.302253 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.302257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.302294 (+ 37us) server_negotiation.cc:407] Connection header received
0619 20:46:51.302319 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.302321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.302343 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.302419 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.303055 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.304010 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.304808 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.305174 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.305230 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.307880 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.307886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.307886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.307893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.309696 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.310060 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.310063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.310063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.310124 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.310317 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.310318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.310318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.310521 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.310690 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.310773 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.310865 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":90,"threads_started":1}
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:51.317534 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.310653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:45755 (local address 127.0.0.1:35172)
0619 20:46:51.310768 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.310779 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.310826 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.310987 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.310988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.311180 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.311236 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.311237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.312011 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.312012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.312490 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.312491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.312871 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.312871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.312939 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.313324 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.313328 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.314551 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.316540 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.316541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.316541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.316686 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.316905 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.316906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.316906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.316963 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.317217 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.317218 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.317261 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.317341 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.317435 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260619 20:46:51.317556 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.310724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35172 (local address 127.13.195.125:45755)
0619 20:46:51.310777 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.310779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.310813 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:46:51.310879 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.310880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.310891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.310943 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.311316 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.311936 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.312584 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.312790 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.312872 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.314645 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.314646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.314646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.314653 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.316233 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.316778 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.316779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.316780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.316836 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.317032 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.317033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.317034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.317150 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.317226 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.317395 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.317472 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:46:51.317962 27668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:46:51.319257 27518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } attempt: 2
I20260619 20:46:51.320335 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.311726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.13.195.126:45243)
0619 20:46:51.311762 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.311765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.311789 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:46:51.311808 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.311808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.311816 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.311860 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.312296 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.312738 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.313574 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.313804 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.313875 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.316326 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.316328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.316328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.316336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.318569 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.318962 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.318963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.318963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.319023 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.319255 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.319256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.319256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.319443 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.320168 (+ 725us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.320205 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.320271 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:46:51.325796 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.318234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50692)
0619 20:46:51.318280 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.318290 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.318327 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.318453 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.318454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.318589 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.318622 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.318623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.319599 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.319600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.320174 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.320176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.320681 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.320682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.320732 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.321110 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.321115 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.322356 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.324774 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.324775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.324775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.324956 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.325145 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.325145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.325146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.325192 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.325441 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.325442 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.325496 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.325562 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.325654 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:46:51.325896 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.318299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.13.195.124:37471)
0619 20:46:51.318336 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.318338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.318346 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:46:51.318370 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.318370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.318380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.318466 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.318696 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.319167 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.320250 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.320562 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.320636 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.322488 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.322489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.322489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.322494 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.324687 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.325025 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.325026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.325026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.325073 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.325249 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.325249 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.325250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.325368 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.325423 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.325598 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.325717 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":23}
I20260619 20:46:51.327841 27668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: Bootstrap starting.
I20260619 20:46:51.328774 27668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:51.329003 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.319497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50696)
0619 20:46:51.319546 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.319556 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.319592 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.320706 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.320707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.320914 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.320957 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.320960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.322055 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.322056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.322838 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.322841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.323387 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.323388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.323469 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.323865 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.323869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.325318 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.327488 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.327489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.327490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.327982 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.328211 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.328212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.328212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.328296 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.328652 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.328653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.328705 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.328763 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.328867 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:46:51.329154 27668 log.cc:824] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:51.329432 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.319543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.13.195.124:37471)
0619 20:46:51.320496 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.320499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.320508 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:46:51.320529 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.320530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.320552 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.320693 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.321073 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.321948 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.322935 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.323300 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.323370 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.325410 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.325412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.325412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.325420 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.327384 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.328072 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.328073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.328073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.328137 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.328375 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.328376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.328377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.328551 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.329128 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.329178 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.329259 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":100,"threads_started":1}
I20260619 20:46:51.329804 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.321006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35186 (local address 127.13.195.125:45755)
0619 20:46:51.321041 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.321043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.321051 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:46:51.321105 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.321107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.321121 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.321157 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.321834 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.322541 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.323560 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.323886 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.323956 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.326827 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.326828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.326828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.326835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.328491 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.328831 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.328833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.328833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.328884 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.329125 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.329126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.329127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.329252 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.329354 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.329609 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.329705 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:46:51.330178 27668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: No bootstrap required, opened a new log
I20260619 20:46:51.331373 27518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930: Bootstrap starting.
I20260619 20:46:51.332002 27518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:51.331944 27668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.332074 27668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:51.332152 27668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06f0a254fa1e4f4a9ba1fc9e6a5196cb, State: Initialized, Role: FOLLOWER
I20260619 20:46:51.332289 27668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.332357 27518 log.cc:824] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:51.332641 27675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.332700 27668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:51.333297 27518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930: No bootstrap required, opened a new log
I20260619 20:46:51.333796 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:46:51.334264 27675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:46:51.335392 27687 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:46:51.335518 27687 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:46:51.335736 27518 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.335866 27518 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:51.335901 27518 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8411d09d0334ba1955e53ba565f1930, State: Initialized, Role: FOLLOWER
I20260619 20:46:51.336009 27518 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.336391 27518 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:51.334928 27600 master_runner.cc:186] Error getting master registration for 127.13.195.126:45243: OK, instance_id { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" instance_seqno: 1781902010974259 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:46:51.337407 27600 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 20:46:51.338202 27707 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:46:51.338316 27707 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:46:51.338683 27693 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.338817 27693 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:46:51.339064 27450 master_runner.cc:186] Error getting master registration for 127.13.195.125:45755: OK, instance_id { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" instance_seqno: 1781902011130501 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:46:51.341925 27450 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:46:51.342082 27693 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:46:51.342226 27693 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.343503 27642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8411d09d0334ba1955e53ba565f1930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" is_pre_election: true
I20260619 20:46:51.343679 27642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8411d09d0334ba1955e53ba565f1930 in term 0.
I20260619 20:46:51.343962 27469 leader_election.cc:302] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06f0a254fa1e4f4a9ba1fc9e6a5196cb, b8411d09d0334ba1955e53ba565f1930; no voters:
I20260619 20:46:51.344230 27693 leader_election.cc:288] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144b8d8ffd674de0804a0ec13a39c349 (127.13.195.125:45755), 06f0a254fa1e4f4a9ba1fc9e6a5196cb (127.13.195.124:37471)
I20260619 20:46:51.344389 27693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:46:51.344532 27693 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:46:51.344641 27693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:51.346143 27693 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.346722 27642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8411d09d0334ba1955e53ba565f1930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb"
I20260619 20:46:51.346791 27642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:51.347635 27642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8411d09d0334ba1955e53ba565f1930 in term 1.
I20260619 20:46:51.347657 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.336557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35200 (local address 127.13.195.125:45755)
0619 20:46:51.336599 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.336602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.339034 (+ 2432us) server_negotiation.cc:407] Connection header received
0619 20:46:51.339134 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.339135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.339147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.339176 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.340034 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.340766 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.342027 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.342368 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.342418 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.344341 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.344342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.344342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.344350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.346228 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.346689 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.346690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.346690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.346752 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.346985 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.346986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.346986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.347129 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.347236 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.347460 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.347547 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:46:51.348026 27469 leader_election.cc:302] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06f0a254fa1e4f4a9ba1fc9e6a5196cb, b8411d09d0334ba1955e53ba565f1930; no voters:
I20260619 20:46:51.348596 27693 leader_election.cc:288] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [CANDIDATE]: Term 1 election: Requested vote from peers 144b8d8ffd674de0804a0ec13a39c349 (127.13.195.125:45755), 06f0a254fa1e4f4a9ba1fc9e6a5196cb (127.13.195.124:37471)
I20260619 20:46:51.348744 27693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:51.348866 27693 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [term 1 LEADER]: Becoming Leader. State: Replica: b8411d09d0334ba1955e53ba565f1930, State: Running, Role: LEADER
I20260619 20:46:51.349028 27693 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.349741 27693 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8411d09d0334ba1955e53ba565f1930. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.349968 27693 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:51.356439 27711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:46:51.357043 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.336899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50700 (local address 127.13.195.124:37471)
0619 20:46:51.336943 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.336945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.347992 (+ 11047us) server_negotiation.cc:407] Connection header received
0619 20:46:51.348103 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.348105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.348119 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.348152 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.348900 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.349754 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.350899 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.351258 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.351308 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.352935 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.352936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.352937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.352944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.355047 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.355494 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.355495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.355496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.355560 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.355793 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.355794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.355794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.355948 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.356040 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.356579 (+ 539us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.356678 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:46:51.358532 27711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:46:51.360467 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.335874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.13.195.126:45243)
0619 20:46:51.335951 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.335954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.337102 (+ 1148us) server_negotiation.cc:407] Connection header received
0619 20:46:51.337240 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.337241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.337253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.337286 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.350885 (+ 13599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.351627 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.353118 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.353415 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.353452 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.356024 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.356025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.356026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.356032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.357687 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.359655 (+ 1968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.359656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.359657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.359710 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.359902 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.359903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.359903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.360019 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.360094 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.360296 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.360374 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:46:51.361248 27642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [term 1 FOLLOWER]: Refusing update from remote peer b8411d09d0334ba1955e53ba565f1930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:51.361615 27693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:51.363102 27675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8411d09d0334ba1955e53ba565f1930. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.363207 27675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:46:51.364516 27709 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.364679 27675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.364780 27675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:46:51.364778 27709 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:51.364919 27712 mvcc.cc:205] Tried to move back new op lower bound from 7298670638530301952 to 7298670638486958080. Current Snapshot: MvccSnapshot[applied={T|T < 7298670638530301952}]
I20260619 20:46:51.365008 27714 mvcc.cc:205] Tried to move back new op lower bound from 7298670638530301952 to 7298670638486958080. Current Snapshot: MvccSnapshot[applied={T|T < 7298670638530301952}]
I20260619 20:46:51.365725 27711 catalog_manager.cc:1347] Generated new cluster ID: 5f71c062d83f43b39afcde0a57c6a4b2
I20260619 20:46:51.365789 27711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:46:51.366998 27593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } attempt: 3
Jun 19 20:46:51 dist-test-slave-n2td krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902011, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:46:51.376461 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.367253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37471 (local address 127.0.0.1:50716)
0619 20:46:51.367342 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:51.367353 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:51.367388 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:51.367637 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:51.367638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:51.367799 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:51.367846 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.367848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.368729 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.368730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.369492 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:51.369494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.369967 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.369968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:51.370027 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.370588 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:51.370594 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:51.372449 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:51.375050 (+ 2601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.375051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.375051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.375320 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.375610 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:51.375612 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:51.375612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:51.375696 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:51.376078 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:51.376079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:51.376150 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 20:46:51.376229 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:46:51.376339 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 20:46:51.377225 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:51.367451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.13.195.124:37471)
0619 20:46:51.367509 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:51.367511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:51.367521 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:46:51.367543 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:51.367544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:51.367555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:51.367633 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:51.367923 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.368649 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.369570 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:51.369846 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:51.369907 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:51.372562 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:51.372563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:51.372564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:51.372571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:51.374919 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.375430 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.375431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.375432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.375511 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:51.375792 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:51.375793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:51.375794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:51.375962 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:51.376026 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:51.377024 (+ 998us) server_negotiation.cc:299] Negotiation successful
0619 20:46:51.377115 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:46:51.378931 27593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349: Bootstrap starting.
I20260619 20:46:51.379755 27593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:51.380105 27593 log.cc:824] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:51.392912 27593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349: No bootstrap required, opened a new log
I20260619 20:46:51.395303 27593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.395465 27593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:51.395501 27593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144b8d8ffd674de0804a0ec13a39c349, State: Initialized, Role: FOLLOWER
I20260619 20:46:51.395931 27593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } }
I20260619 20:46:51.396937 27565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:51.398016 27565 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [term 1 FOLLOWER]: Refusing update from remote peer b8411d09d0334ba1955e53ba565f1930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:51.398396 27566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8411d09d0334ba1955e53ba565f1930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144b8d8ffd674de0804a0ec13a39c349"
I20260619 20:46:51.398391 27713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:51.399251 27566 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8411d09d0334ba1955e53ba565f1930 in term 1.
I20260619 20:46:51.399607 27567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8411d09d0334ba1955e53ba565f1930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144b8d8ffd674de0804a0ec13a39c349" is_pre_election: true
I20260619 20:46:51.399974 27593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:51.406559 27723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.406330 27711 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:46:51.406708 27723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:46:51.406774 27723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8411d09d0334ba1955e53ba565f1930. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.406816 27723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:46:51.406849 27723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.406890 27723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:46:51.407672 27713 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8411d09d0334ba1955e53ba565f1930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8411d09d0334ba1955e53ba565f1930" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45243 } } peers { permanent_uuid: "144b8d8ffd674de0804a0ec13a39c349" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 45755 } } peers { permanent_uuid: "06f0a254fa1e4f4a9ba1fc9e6a5196cb" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37471 } } }
I20260619 20:46:51.407895 27721 mvcc.cc:205] Tried to move back new op lower bound from 7298670638530301952 to 7298670638486958080. Current Snapshot: MvccSnapshot[applied={T|T < 7298670638530301952}]
I20260619 20:46:51.407922 27713 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930 [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:51.407428 27711 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:46:51.413079 27734 catalog_manager.cc:1259] Loaded cluster ID: 5f71c062d83f43b39afcde0a57c6a4b2
I20260619 20:46:51.413235 27734 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349: loading cluster ID for follower catalog manager: success
I20260619 20:46:51.414692 27734 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349: acquiring CA information for follower catalog manager: success
W20260619 20:46:51.414907 27734 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260619 20:46:51.416081 27711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8411d09d0334ba1955e53ba565f1930: Generated new TSK 0
I20260619 20:46:51.416375 27711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:46:52.336030 27687 catalog_manager.cc:1259] Loaded cluster ID: 5f71c062d83f43b39afcde0a57c6a4b2
I20260619 20:46:52.336108 27687 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: loading cluster ID for follower catalog manager: success
I20260619 20:46:52.337107 27687 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: acquiring CA information for follower catalog manager: success
I20260619 20:46:52.337496 27687 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 06f0a254fa1e4f4a9ba1fc9e6a5196cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:46:52.415601 27734 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 144b8d8ffd674de0804a0ec13a39c349: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:46:52.475490 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27450
I20260619 20:46:52.480736 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27525
I20260619 20:46:52.485370 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27600
2026-06-19T20:46:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2809 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:46:52 dist-test-slave-n2td krb5kdc[27745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:46:52 dist-test-slave-n2td krb5kdc[27745](info): set up 2 sockets
Jun 19 20:46:52 dist-test-slave-n2td krb5kdc[27745](info): commencing operation
krb5kdc: starting...
W20260619 20:46:53.533679 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:46:53 dist-test-slave-n2td krb5kdc[27745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:46:53.662650 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:35243
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:53.743523 27758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:53.743788 27758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:53.743826 27758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:53.745007 27758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:53.745085 27758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:53.745110 27758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:53.745131 27758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:53.745158 27758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:53.746655 27758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:53.746932 27758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:53.747172 27758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:53.752146 27766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:53.752215 27763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:53.752229 27758 server_base.cc:1034] running on GCE node
W20260619 20:46:53.752146 27764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:53.752594 27758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:53.752882 27758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:53.754043 27758 hybrid_clock.cc:648] HybridClock initialized: now 1781902013754009 us; error 72 us; skew 500 ppm
Jun 19 20:46:53 dist-test-slave-n2td krb5kdc[27745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902013, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:53.756480 27758 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:46:53.756938 27758 webserver.cc:460] Webserver started at http://127.13.195.126:42293/ using document root <none> and password file <none>
I20260619 20:46:53.757157 27758 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:53.757231 27758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:53.757381 27758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:53.758503 27758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "782b4f3d441f44708f065676dfe2d02d"
format_stamp: "Formatted at 2026-06-19 20:46:53 on dist-test-slave-n2td"
server_key: "8e3a5b7b42def315e292294877274dc5"
server_key_iv: "ab6c39dd66aa5635e5cc28b03f1a1efa"
server_key_version: "encryptionkey@0"
I20260619 20:46:53.758872 27758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "782b4f3d441f44708f065676dfe2d02d"
format_stamp: "Formatted at 2026-06-19 20:46:53 on dist-test-slave-n2td"
server_key: "8e3a5b7b42def315e292294877274dc5"
server_key_iv: "ab6c39dd66aa5635e5cc28b03f1a1efa"
server_key_version: "encryptionkey@0"
I20260619 20:46:53.760229 27758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:53.760962 27773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:53.761253 27758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:53.761389 27758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "782b4f3d441f44708f065676dfe2d02d"
format_stamp: "Formatted at 2026-06-19 20:46:53 on dist-test-slave-n2td"
server_key: "8e3a5b7b42def315e292294877274dc5"
server_key_iv: "ab6c39dd66aa5635e5cc28b03f1a1efa"
server_key_version: "encryptionkey@0"
I20260619 20:46:53.761482 27758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:53.769976 27758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:53.772351 27758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:53.772523 27758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:53.776845 27758 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:35243
I20260619 20:46:53.776901 27825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:35243 every 8 connection(s)
I20260619 20:46:53.777287 27758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:46:53.777763 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27758
I20260619 20:46:53.777889 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:46:53.778144 14093 external_mini_cluster.cc:1428] Setting key a410715168f4d93fc8b803625d0d67ef
I20260619 20:46:53.778483 27826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:53.781410 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d: Bootstrap starting.
I20260619 20:46:53.782034 27826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:53.782419 27826 log.cc:824] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:53.783257 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d: No bootstrap required, opened a new log
Jun 19 20:46:53 dist-test-slave-n2td krb5kdc[27745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:53.785491 27826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782b4f3d441f44708f065676dfe2d02d" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35243 } }
I20260619 20:46:53.785672 27826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:53.785701 27826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782b4f3d441f44708f065676dfe2d02d, State: Initialized, Role: FOLLOWER
I20260619 20:46:53.785876 27826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782b4f3d441f44708f065676dfe2d02d" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35243 } }
I20260619 20:46:53.785965 27826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:46:53.786000 27826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:46:53.786058 27826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:53.786741 27826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782b4f3d441f44708f065676dfe2d02d" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35243 } }
I20260619 20:46:53.786831 27826 leader_election.cc:302] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 782b4f3d441f44708f065676dfe2d02d; no voters:
I20260619 20:46:53.787035 27826 leader_election.cc:288] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:46:53.787081 27831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:53.787248 27831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [term 1 LEADER]: Becoming Leader. State: Replica: 782b4f3d441f44708f065676dfe2d02d, State: Running, Role: LEADER
I20260619 20:46:53.787317 27826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:53.787396 27831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782b4f3d441f44708f065676dfe2d02d" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35243 } }
I20260619 20:46:53.787719 27831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "782b4f3d441f44708f065676dfe2d02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782b4f3d441f44708f065676dfe2d02d" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35243 } } }
I20260619 20:46:53.787797 27831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:53.787923 27831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 782b4f3d441f44708f065676dfe2d02d. Latest consensus state: current_term: 1 leader_uuid: "782b4f3d441f44708f065676dfe2d02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782b4f3d441f44708f065676dfe2d02d" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35243 } } }
I20260619 20:46:53.787984 27831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:53.788218 27838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:46:53.789485 27838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:46:53.789487 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:53.778695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.13.195.126:35243)
0619 20:46:53.778875 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:53.778886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:53.778921 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:46:53.779592 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:53.779599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:53.779682 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:53.779837 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:53.780476 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:53.781370 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:53.782214 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:53.782843 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:53.782985 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:53.785576 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:53.785580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:53.785580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:53.785590 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:53.787701 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:53.788139 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:53.788142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:53.788142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:53.788283 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:53.788491 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:53.788492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:53.788492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:53.788809 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:53.788981 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:53.789121 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 20:46:53.789284 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260619 20:46:53.790534 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:46:53.791262 27838 catalog_manager.cc:1347] Generated new cluster ID: 47aa72b1e48b465fbb1a02a00277a0e4
I20260619 20:46:53.791316 27838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:46:53.818593 27838 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:46:53.819384 27838 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:46:53.832909 27838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 782b4f3d441f44708f065676dfe2d02d: Generated new TSK 0
I20260619 20:46:53.833138 27838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:46:53.844617 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27758
2026-06-19T20:46:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1357 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:46:53 dist-test-slave-n2td krb5kdc[27853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:46:53 dist-test-slave-n2td krb5kdc[27853](info): set up 2 sockets
Jun 19 20:46:53 dist-test-slave-n2td krb5kdc[27853](info): commencing operation
krb5kdc: starting...
W20260619 20:46:54.902218 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:46:54 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:46:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:46:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:46:55.022496 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44785
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:55.103391 27866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:55.103638 27866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:55.103683 27866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:55.104815 27866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:55.104885 27866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:55.104909 27866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:55.104938 27866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:55.104967 27866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:55.106395 27866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:55.106698 27866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:55.106925 27866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:55.111852 27872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.111924 27871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.111948 27874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:55.112123 27866 server_base.cc:1034] running on GCE node
I20260619 20:46:55.112550 27866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:55.112831 27866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:55.113993 27866 hybrid_clock.cc:648] HybridClock initialized: now 1781902015113981 us; error 46 us; skew 500 ppm
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902015, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:55.116427 27866 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:46:55.116899 27866 webserver.cc:460] Webserver started at http://127.13.195.126:36611/ using document root <none> and password file <none>
I20260619 20:46:55.117089 27866 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:55.117128 27866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:55.117216 27866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:55.118171 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "051d3ad8f83647c1a5b78f34f4b75b8c"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "76f89210f3eaacf6100debfd8698f072"
server_key_iv: "e48422d4ca3b4444e3ba4f62b05def11"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.118525 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "051d3ad8f83647c1a5b78f34f4b75b8c"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "76f89210f3eaacf6100debfd8698f072"
server_key_iv: "e48422d4ca3b4444e3ba4f62b05def11"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.119761 27866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:55.120419 27881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.120568 27866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.120644 27866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "051d3ad8f83647c1a5b78f34f4b75b8c"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "76f89210f3eaacf6100debfd8698f072"
server_key_iv: "e48422d4ca3b4444e3ba4f62b05def11"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.120698 27866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:55.144321 27866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:55.146517 27866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:55.146648 27866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:55.150925 27866 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:44785
I20260619 20:46:55.151000 27933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:44785 every 8 connection(s)
I20260619 20:46:55.151393 27866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:46:55.152401 27934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:55.154685 27934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Bootstrap starting.
I20260619 20:46:55.155259 27934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:55.155544 27934 log.cc:824] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:55.156234 27934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: No bootstrap required, opened a new log
I20260619 20:46:55.157519 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27866
I20260619 20:46:55.157642 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:46:55.157841 14093 external_mini_cluster.cc:1428] Setting key 5cd2b83ad9c086dc3a27c1d7acb2da58
I20260619 20:46:55.157902 27934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:55.158098 27934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:55.158139 27934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d3ad8f83647c1a5b78f34f4b75b8c, State: Initialized, Role: FOLLOWER
I20260619 20:46:55.158309 27934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:55.158407 27934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:46:55.158476 27934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:46:55.158535 27934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:55.159528 27934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:55.159663 27934 leader_election.cc:302] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 051d3ad8f83647c1a5b78f34f4b75b8c; no voters:
I20260619 20:46:55.159803 27934 leader_election.cc:288] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:46:55.159870 27939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:55.160006 27939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 LEADER]: Becoming Leader. State: Replica: 051d3ad8f83647c1a5b78f34f4b75b8c, State: Running, Role: LEADER
I20260619 20:46:55.160027 27934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:55.160090 27939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:55.160396 27940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } } }
I20260619 20:46:55.160485 27940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:55.160386 27939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 051d3ad8f83647c1a5b78f34f4b75b8c. Latest consensus state: current_term: 1 leader_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } } }
I20260619 20:46:55.160681 27939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:55.160804 27947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:46:55.161430 27947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:46:55.163089 27947 catalog_manager.cc:1347] Generated new cluster ID: 188e3420e7d84159844a80eaa6f7beac
I20260619 20:46:55.163138 27947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:55.169317 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.158374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45460 (local address 127.13.195.126:44785)
0619 20:46:55.158582 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.158591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.158626 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:46:55.158808 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.158816 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.158892 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.159056 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:55.159646 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.161095 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.162142 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.162722 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.162868 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.165578 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:55.165582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:55.165582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:55.165593 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:55.167573 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.168000 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.168002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.168003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.168111 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.168351 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.168351 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.168352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.168645 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:55.168800 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.168993 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.169123 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260619 20:46:55.185111 27947 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:46:55.185748 27947 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:46:55.189402 27947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Generated new TSK 0
I20260619 20:46:55.189582 27947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:46:55.219388 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:44785
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:55.303226 27960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:55.303445 27960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:55.303470 27960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:55.304563 27960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:55.304598 27960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:55.304625 27960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:46:55.305992 27960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:55.306183 27960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:55.306360 27960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:55.311435 27968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.311468 27965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.311476 27966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:55.311628 27960 server_base.cc:1034] running on GCE node
I20260619 20:46:55.311833 27960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:55.312088 27960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:55.313262 27960 hybrid_clock.cc:648] HybridClock initialized: now 1781902015313259 us; error 39 us; skew 500 ppm
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902015, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:55.315980 27960 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:46:55.316447 27960 webserver.cc:460] Webserver started at http://127.13.195.65:33831/ using document root <none> and password file <none>
I20260619 20:46:55.316694 27960 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:55.316763 27960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:55.316915 27960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:55.318023 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "b48f0dc31d0c4887a36e875e1ed0ec06"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "9870a0de7ff65be70c1a648b8d9f2551"
server_key_iv: "ce2ac04b3e2610e526602751c36ef296"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.318385 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "b48f0dc31d0c4887a36e875e1ed0ec06"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "9870a0de7ff65be70c1a648b8d9f2551"
server_key_iv: "ce2ac04b3e2610e526602751c36ef296"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.319839 27960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:55.320619 27975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.320798 27960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:46:55.320897 27960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "b48f0dc31d0c4887a36e875e1ed0ec06"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "9870a0de7ff65be70c1a648b8d9f2551"
server_key_iv: "ce2ac04b3e2610e526602751c36ef296"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.320982 27960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:55.329147 27960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:55.331004 27960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:55.331167 27960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:55.331388 27960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:55.331715 27960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:55.331777 27960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.331820 27960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:55.331846 27960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.337882 27960 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:34507
I20260619 20:46:55.337931 28088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:34507 every 8 connection(s)
I20260619 20:46:55.338253 27960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:46:55.344547 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 27960
I20260619 20:46:55.344666 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902015, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:55.344871 14093 external_mini_cluster.cc:1428] Setting key b25a8af455dc71cd26304ea1a7b50f7b
I20260619 20:46:55.348486 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.339007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:38221 (local address 127.13.195.126:44785)
0619 20:46:55.339087 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.339089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.339339 (+ 250us) server_negotiation.cc:407] Connection header received
0619 20:46:55.339868 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.339869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.339884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.339951 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:55.340990 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.341507 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.342624 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.342829 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.342894 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.345268 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:55.345269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:55.345269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:55.345278 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:55.346962 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.347422 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.347423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.347424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.347492 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.347736 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.347736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.347737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.347889 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:55.347964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.348233 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.348344 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:46:55.348517 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.338980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:38221)
0619 20:46:55.339176 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:55.339223 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:55.339748 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:55.340036 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:55.340040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:55.340670 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:55.340888 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.340893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.341595 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.341595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.342494 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.342497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.342924 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.342925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.343035 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.343532 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:55.343540 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:55.345126 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:55.347046 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:55.347053 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:55.347055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:55.347344 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:55.347566 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:55.347567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:55.347567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:55.347672 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:55.347960 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:55.347965 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:55.348092 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 20:46:55.348197 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 20:46:55.348341 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260619 20:46:55.349040 28089 heartbeater.cc:347] Connected to a master server at 127.13.195.126:44785
I20260619 20:46:55.349136 28089 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:55.349402 28089 heartbeater.cc:511] Master 127.13.195.126:44785 requested a full tablet report, sending...
I20260619 20:46:55.350047 27898 ts_manager.cc:194] Registered new tserver with Master: b48f0dc31d0c4887a36e875e1ed0ec06 (127.13.195.65:34507)
I20260619 20:46:55.350756 27898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:38221
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:46:55.369500 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:44785
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:55.453608 28094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:55.453826 28094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:55.453850 28094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:55.455370 28094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:55.455433 28094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:55.455466 28094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:46:55.457224 28094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:55.457437 28094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:55.457651 28094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:55.463117 28100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.463130 28102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.463140 28099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:55.463889 28094 server_base.cc:1034] running on GCE node
I20260619 20:46:55.464129 28094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:55.464375 28094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:55.465543 28094 hybrid_clock.cc:648] HybridClock initialized: now 1781902015465540 us; error 37 us; skew 500 ppm
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902015, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:55.467991 28094 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:46:55.468482 28094 webserver.cc:460] Webserver started at http://127.13.195.66:40225/ using document root <none> and password file <none>
I20260619 20:46:55.468729 28094 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:55.468794 28094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:55.468937 28094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:55.469980 28094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "70e58bfbc17048e9a52605640790f5b4"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "7df8352ff37ce336fce5c9e497f4893b"
server_key_iv: "eb3eceabad2beb53fbc849c11db27bf3"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.470340 28094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "70e58bfbc17048e9a52605640790f5b4"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "7df8352ff37ce336fce5c9e497f4893b"
server_key_iv: "eb3eceabad2beb53fbc849c11db27bf3"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.471657 28094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:55.472406 28109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.472577 28094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:46:55.472692 28094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "70e58bfbc17048e9a52605640790f5b4"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "7df8352ff37ce336fce5c9e497f4893b"
server_key_iv: "eb3eceabad2beb53fbc849c11db27bf3"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.472765 28094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:55.481560 28094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:55.483368 28094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:55.483523 28094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:55.483749 28094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:55.484045 28094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:55.484097 28094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.484138 28094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:55.484165 28094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.490116 28094 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:45557
I20260619 20:46:55.490176 28222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:45557 every 8 connection(s)
I20260619 20:46:55.490551 28094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:46:55.494724 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28094
I20260619 20:46:55.494863 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:46:55.495062 14093 external_mini_cluster.cc:1428] Setting key 57d21f05d956c91cd6cfe3cebddea311
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902015, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:55.500612 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.491423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:53011 (local address 127.13.195.126:44785)
0619 20:46:55.491476 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.491478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.491625 (+ 147us) server_negotiation.cc:407] Connection header received
0619 20:46:55.492109 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.492110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.492120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.492180 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:55.493095 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.493589 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.494661 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.494890 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.494961 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.497548 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:55.497550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:55.497550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:55.497559 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:55.499051 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.499604 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.499606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.499606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.499679 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.499983 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.499983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.499984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.500085 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:55.500136 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.500481 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.500547 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:46:55.500986 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.491337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:53011)
0619 20:46:55.491491 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:55.491533 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:55.492022 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:55.492244 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:55.492247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:55.492810 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:55.493005 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.493009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.493685 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.493685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.494565 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.494568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.494969 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.494970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.495125 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.495800 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:55.495824 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:55.497428 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:55.499138 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:55.499145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:55.499147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:55.499486 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:55.499757 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:55.499758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:55.499758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:55.499914 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:55.500151 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:55.500158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:55.500337 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 20:46:55.500467 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 20:46:55.500671 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260619 20:46:55.501595 28223 heartbeater.cc:347] Connected to a master server at 127.13.195.126:44785
I20260619 20:46:55.501714 28223 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:55.501933 28223 heartbeater.cc:511] Master 127.13.195.126:44785 requested a full tablet report, sending...
I20260619 20:46:55.502369 27898 ts_manager.cc:194] Registered new tserver with Master: 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557)
I20260619 20:46:55.502770 27898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:53011
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:46:55.520857 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:44785
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:55.604218 28228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:55.604488 28228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:55.604532 28228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:55.605952 28228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:55.606005 28228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:55.606033 28228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:46:55.607503 28228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:55.607722 28228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:55.607908 28228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:55.613088 28234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.613148 28236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:55.613077 28233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:55.613363 28228 server_base.cc:1034] running on GCE node
I20260619 20:46:55.613600 28228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:55.613916 28228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:55.615056 28228 hybrid_clock.cc:648] HybridClock initialized: now 1781902015615033 us; error 39 us; skew 500 ppm
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902015, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:55.617614 28228 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:46:55.618072 28228 webserver.cc:460] Webserver started at http://127.13.195.67:39623/ using document root <none> and password file <none>
I20260619 20:46:55.618299 28228 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:55.618366 28228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:55.618520 28228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:46:55.619560 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "0a7d0ef9783a70113ffb7c212fee21ca"
server_key_iv: "12cbaf6dad199f847b5c5fd7cb28075b"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.619930 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "0a7d0ef9783a70113ffb7c212fee21ca"
server_key_iv: "12cbaf6dad199f847b5c5fd7cb28075b"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.621255 28228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:46:55.622001 28243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.622174 28228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:46:55.622268 28228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "0a7d0ef9783a70113ffb7c212fee21ca"
server_key_iv: "12cbaf6dad199f847b5c5fd7cb28075b"
server_key_version: "encryptionkey@0"
I20260619 20:46:55.622351 28228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:55.638197 28228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:55.640033 28228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:55.640190 28228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:55.640424 28228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:55.640733 28228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:55.640784 28228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.640825 28228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:55.640852 28228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:55.646538 28228 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:35063
I20260619 20:46:55.646601 28356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:35063 every 8 connection(s)
I20260619 20:46:55.646924 28228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:46:55 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902015, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:55.655632 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.647640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:37855 (local address 127.13.195.126:44785)
0619 20:46:55.647725 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.647728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.647913 (+ 185us) server_negotiation.cc:407] Connection header received
0619 20:46:55.648395 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.648397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.648406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.648458 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:55.649344 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.649872 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.650910 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.651103 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.651172 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.653290 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:55.653291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:55.653291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:55.653298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:55.654504 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.654871 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.654872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.654872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.654922 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.655123 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.655124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.655124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.655228 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:55.655279 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.655503 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.655563 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 20:46:55.655815 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.647611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:37855)
0619 20:46:55.647798 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:55.647834 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:55.648314 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:55.648508 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:55.648511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:55.649056 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:55.649245 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.649249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.649954 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.649955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.650833 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.650835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.651188 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.651189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.651306 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.651873 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:55.651882 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:55.653207 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:55.654578 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:55.654583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:55.654585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:55.654808 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:55.654979 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:55.654979 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:55.654979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:55.655070 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:55.655282 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:55.655286 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:55.655396 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 20:46:55.655475 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:46:55.655591 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260619 20:46:55.655997 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28228
I20260619 20:46:55.656113 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:46:55.656286 28357 heartbeater.cc:347] Connected to a master server at 127.13.195.126:44785
I20260619 20:46:55.656325 14093 external_mini_cluster.cc:1428] Setting key 205724d352105a3b15d1560b05c40be0
I20260619 20:46:55.656399 28357 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:55.656595 28357 heartbeater.cc:511] Master 127.13.195.126:44785 requested a full tablet report, sending...
I20260619 20:46:55.658125 27898 ts_manager.cc:194] Registered new tserver with Master: 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063)
I20260619 20:46:55.658586 27898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:37855
I20260619 20:46:55.668118 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:46:55.675485 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.669455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.13.195.126:44785)
0619 20:46:55.669516 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.669517 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.669604 (+ 87us) server_negotiation.cc:407] Connection header received
0619 20:46:55.669643 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.669644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.669652 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.669713 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:55.670192 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.670758 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.671441 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.671720 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.671774 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.672846 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:55.672847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:55.672847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:55.672853 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:55.674190 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.674635 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.674636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.674637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.674732 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:55.674938 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:55.674939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:55.674939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:55.675084 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:55.675191 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.675317 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.675389 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:46:55.677155 27898 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:46:55.678834 27898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:46:55.685218 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.681053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:35063 (local address 127.0.0.1:42138)
0619 20:46:55.681359 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:55.681404 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:55.681481 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:55.681773 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:55.681777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:55.681779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:55.681840 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.681842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.683224 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.683224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.684214 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.684217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.684902 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.684902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.684978 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.684991 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:46:55.685059 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:46:55.685114 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":35,"thread_start_us":61,"threads_started":1}
I20260619 20:46:55.685322 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.681358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36026 (local address 127.13.195.66:45557)
0619 20:46:55.681541 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.681545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.681696 (+ 151us) server_negotiation.cc:407] Connection header received
0619 20:46:55.681734 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.681736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.681753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.681825 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:55.682088 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.682851 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.684227 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.684956 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.685037 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.685089 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.685196 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.685236 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260619 20:46:55.685526 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.681054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:34507 (local address 127.0.0.1:33378)
0619 20:46:55.681525 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:55.681536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:55.681593 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:55.681903 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:55.681904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:55.681909 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:55.681989 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.681990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.683279 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.683279 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.684349 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.684351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.685281 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.685282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.685357 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.685361 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:46:55.685414 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:46:55.685451 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":26,"spinlock_wait_cycles":768,"thread_start_us":56,"threads_started":1}
I20260619 20:46:55.685539 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.681153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.13.195.67:35063)
0619 20:46:55.681488 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.681492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.681504 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:46:55.681588 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.681591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.681610 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.681691 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:55.681918 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.683106 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.684284 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.684816 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.684903 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.684962 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.685425 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.685465 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":115,"threads_started":1}
I20260619 20:46:55.685827 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.681261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:45557 (local address 127.0.0.1:36026)
0619 20:46:55.681589 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:55.681598 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:55.681642 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:55.681904 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:55.681904 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:55.681906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:55.681989 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.681990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.682947 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.682948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.684135 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:55.684138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.685039 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.685040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:55.685107 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.685110 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:46:55.685165 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:46:55.685203 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":5,"thread_start_us":36,"threads_started":1}
I20260619 20:46:55.685870 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:55.681155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.13.195.65:34507)
0619 20:46:55.681401 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:55.681406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:55.681590 (+ 184us) server_negotiation.cc:407] Connection header received
0619 20:46:55.681735 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:55.681737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:55.681753 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:55.681824 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:55.682075 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.683182 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.684428 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:55.685133 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:55.685197 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:55.685246 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:55.685728 (+ 482us) server_negotiation.cc:299] Negotiation successful
0619 20:46:55.685771 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":117,"threads_started":1}
I20260619 20:46:55.686296 28291 tablet_service.cc:1467] Processing CreateTablet for tablet 149e740e79434d5794044d62740fc99f (DEFAULT_TABLE table=test-table [id=92f156e90fd54873b5db29170ab043b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:55.686395 28023 tablet_service.cc:1467] Processing CreateTablet for tablet 149e740e79434d5794044d62740fc99f (DEFAULT_TABLE table=test-table [id=92f156e90fd54873b5db29170ab043b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:55.686555 28157 tablet_service.cc:1467] Processing CreateTablet for tablet 149e740e79434d5794044d62740fc99f (DEFAULT_TABLE table=test-table [id=92f156e90fd54873b5db29170ab043b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:55.686691 28023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 149e740e79434d5794044d62740fc99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:55.686691 28291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 149e740e79434d5794044d62740fc99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:55.686899 28157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 149e740e79434d5794044d62740fc99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:55.689517 28371 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Bootstrap starting.
I20260619 20:46:55.689880 28372 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Bootstrap starting.
I20260619 20:46:55.690160 28373 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Bootstrap starting.
I20260619 20:46:55.690336 28371 tablet_bootstrap.cc:654] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:55.690901 28373 tablet_bootstrap.cc:654] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:55.690943 28371 log.cc:824] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:55.691511 28373 log.cc:824] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:55.691833 28371 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: No bootstrap required, opened a new log
I20260619 20:46:55.691857 28372 tablet_bootstrap.cc:654] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:55.691922 28371 ts_tablet_manager.cc:1397] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:46:55.692390 28373 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: No bootstrap required, opened a new log
I20260619 20:46:55.692436 28372 log.cc:824] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:55.692510 28373 ts_tablet_manager.cc:1397] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:46:55.693248 28372 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: No bootstrap required, opened a new log
I20260619 20:46:55.693320 28372 ts_tablet_manager.cc:1397] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:46:55.693894 28371 raft_consensus.cc:348] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:55.694072 28371 raft_consensus.cc:374] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:55.694123 28371 raft_consensus.cc:729] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e58bfbc17048e9a52605640790f5b4, State: Initialized, Role: FOLLOWER
I20260619 20:46:55.694247 28371 consensus_queue.cc:260] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:55.694583 28371 ts_tablet_manager.cc:1428] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 20:46:55.694532 28373 raft_consensus.cc:348] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:55.694703 28373 raft_consensus.cc:374] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:55.694739 28373 raft_consensus.cc:729] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48f0dc31d0c4887a36e875e1ed0ec06, State: Initialized, Role: FOLLOWER
I20260619 20:46:55.694737 28223 heartbeater.cc:503] Master 127.13.195.126:44785 was elected leader, sending a full tablet report...
I20260619 20:46:55.694846 28373 consensus_queue.cc:260] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:55.695101 28373 ts_tablet_manager.cc:1428] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:46:55.695218 28089 heartbeater.cc:503] Master 127.13.195.126:44785 was elected leader, sending a full tablet report...
I20260619 20:46:55.695365 28372 raft_consensus.cc:348] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:55.695518 28372 raft_consensus.cc:374] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:55.695560 28372 raft_consensus.cc:729] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef3f492a39d457a8bcfaf9f7fd1a0ce, State: Initialized, Role: FOLLOWER
I20260619 20:46:55.695669 28372 consensus_queue.cc:260] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:55.695972 28372 ts_tablet_manager.cc:1428] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:46:55.696010 28357 heartbeater.cc:503] Master 127.13.195.126:44785 was elected leader, sending a full tablet report...
W20260619 20:46:55.741150 28224 tablet.cc:2369] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:46:55.839001 28090 tablet.cc:2369] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:46:55.897512 28358 tablet.cc:2369] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:46:56.058717 28378 raft_consensus.cc:482] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:46:56.058838 28378 raft_consensus.cc:504] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:56.059252 28378 leader_election.cc:288] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063), 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557)
I20260619 20:46:56.063529 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.059580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:47651 (local address 127.13.195.67:35063)
0619 20:46:56.059688 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.059690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.059759 (+ 69us) server_negotiation.cc:407] Connection header received
0619 20:46:56.059826 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.059827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.059838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.059916 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:56.060144 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.060903 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.062584 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.063167 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.063241 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.063249 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.063417 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.063458 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 20:46:56.063529 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.059805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:39371 (local address 127.13.195.66:45557)
0619 20:46:56.059846 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.059848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.059859 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:46:56.059900 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.059901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.059910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.059960 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:56.060129 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.060908 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.062585 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.063159 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.063241 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.063249 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.063417 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.063458 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:46:56.063583 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.059485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:35063 (local address 127.13.195.65:47651)
0619 20:46:56.059677 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.059689 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.059762 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.059931 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.059932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.059953 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:56.060052 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.060057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.060998 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.060999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.062477 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.062481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.063249 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.063250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.063334 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.063340 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.063399 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.063459 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":92,"thread_start_us":91,"threads_started":1}
I20260619 20:46:56.063587 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.059687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:45557 (local address 127.13.195.65:39371)
0619 20:46:56.059793 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.059802 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.059840 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.059972 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.059972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.059974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:56.060052 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.060054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.061007 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.061007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.062477 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.062481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.063251 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.063252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.063334 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.063338 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.063399 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.063459 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":71,"thread_start_us":46,"threads_started":1}
I20260619 20:46:56.064127 28311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149e740e79434d5794044d62740fc99f" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" is_pre_election: true
I20260619 20:46:56.064314 28311 raft_consensus.cc:2454] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 0.
I20260619 20:46:56.064592 27979 leader_election.cc:302] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ef3f492a39d457a8bcfaf9f7fd1a0ce, b48f0dc31d0c4887a36e875e1ed0ec06; no voters:
I20260619 20:46:56.064617 28177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149e740e79434d5794044d62740fc99f" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e58bfbc17048e9a52605640790f5b4" is_pre_election: true
I20260619 20:46:56.064736 28378 raft_consensus.cc:2781] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:46:56.064780 28177 raft_consensus.cc:2454] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 0.
I20260619 20:46:56.064795 28378 raft_consensus.cc:482] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:46:56.064837 28378 raft_consensus.cc:3037] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:56.065586 28378 raft_consensus.cc:504] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:56.065721 28378 leader_election.cc:288] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 election: Requested vote from peers 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063), 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557)
I20260619 20:46:56.065871 28311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149e740e79434d5794044d62740fc99f" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce"
I20260619 20:46:56.065889 28177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "149e740e79434d5794044d62740fc99f" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e58bfbc17048e9a52605640790f5b4"
I20260619 20:46:56.065971 28311 raft_consensus.cc:3037] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:56.065989 28177 raft_consensus.cc:3037] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:56.066777 28311 raft_consensus.cc:2454] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 1.
I20260619 20:46:56.066797 28177 raft_consensus.cc:2454] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 1.
I20260619 20:46:56.066937 27979 leader_election.cc:302] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e58bfbc17048e9a52605640790f5b4, b48f0dc31d0c4887a36e875e1ed0ec06; no voters:
I20260619 20:46:56.067124 28378 raft_consensus.cc:2781] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:56.067279 28378 raft_consensus.cc:686] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 LEADER]: Becoming Leader. State: Replica: b48f0dc31d0c4887a36e875e1ed0ec06, State: Running, Role: LEADER
I20260619 20:46:56.067400 28378 consensus_queue.cc:237] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:56.068226 27898 catalog_manager.cc:5495] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 reported cstate change: term changed from 0 to 1, leader changed from <none> to b48f0dc31d0c4887a36e875e1ed0ec06 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } health_report { overall_health: UNKNOWN } } }
I20260619 20:46:56.124866 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.122183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.13.195.65:34507)
0619 20:46:56.122238 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.122240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.122248 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:46:56.122316 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.122317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.122335 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.122395 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:46:56.122546 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.123116 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.123925 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.124103 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.124210 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.124482 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:46:56.124585 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:46:56.124651 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.124721 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.124763 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:46:56.126646 28311 raft_consensus.cc:1261] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Refusing update from remote peer b48f0dc31d0c4887a36e875e1ed0ec06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:56.126775 28177 raft_consensus.cc:1261] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Refusing update from remote peer b48f0dc31d0c4887a36e875e1ed0ec06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:56.127054 28382 consensus_queue.cc:1035] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:56.127220 28378 consensus_queue.cc:1035] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:56.129786 28387 mvcc.cc:205] Tried to move back new op lower bound from 7298670658051817472 to 7298670657812606976. Current Snapshot: MvccSnapshot[applied={T|T < 7298670658051817472}]
I20260619 20:46:56.130242 28386 mvcc.cc:205] Tried to move back new op lower bound from 7298670658051817472 to 7298670657812606976. Current Snapshot: MvccSnapshot[applied={T|T < 7298670658051817472 or (T in {7298670658051817472})}]
I20260619 20:46:56.132140 27898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:56.132251 27898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:56.133074 27898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Sending DeleteTablet for 3 replicas of tablet 149e740e79434d5794044d62740fc99f
I20260619 20:46:56.133405 28023 tablet_service.cc:1514] Processing DeleteTablet for tablet 149e740e79434d5794044d62740fc99f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:56 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:33378
I20260619 20:46:56.133504 28291 tablet_service.cc:1514] Processing DeleteTablet for tablet 149e740e79434d5794044d62740fc99f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:56 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:42138
I20260619 20:46:56.133648 28395 tablet_replica.cc:330] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: stopping tablet replica
I20260619 20:46:56.133766 28395 raft_consensus.cc:2229] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:46:56.133769 28396 tablet_replica.cc:330] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: stopping tablet replica
I20260619 20:46:56.133744 28157 tablet_service.cc:1514] Processing DeleteTablet for tablet 149e740e79434d5794044d62740fc99f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:56 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:36026
I20260619 20:46:56.133898 28395 raft_consensus.cc:2258] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:56.133927 28396 raft_consensus.cc:2229] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:46:56.134068 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27960
I20260619 20:46:56.134325 28395 ts_tablet_manager.cc:1905] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:56.135123 28396 raft_consensus.cc:2258] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:56.135545 28396 ts_tablet_manager.cc:1905] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:56.136389 28397 tablet_replica.cc:330] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: stopping tablet replica
I20260619 20:46:56.136508 28397 raft_consensus.cc:2229] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:46:56.136623 28397 raft_consensus.cc:2258] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:56.137038 28396 ts_tablet_manager.cc:1918] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:56.137073 28397 ts_tablet_manager.cc:1905] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:56.137125 28396 log.cc:1196] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/149e740e79434d5794044d62740fc99f
I20260619 20:46:56.137362 28396 ts_tablet_manager.cc:1939] T 149e740e79434d5794044d62740fc99f P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Deleting consensus metadata
I20260619 20:46:56.137755 27885 catalog_manager.cc:4841] TS 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063): tablet 149e740e79434d5794044d62740fc99f (table test-table [id=92f156e90fd54873b5db29170ab043b1]) successfully deleted
I20260619 20:46:56.139024 28397 ts_tablet_manager.cc:1918] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:56.139102 28397 log.cc:1196] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/149e740e79434d5794044d62740fc99f
I20260619 20:46:56.139380 28397 ts_tablet_manager.cc:1939] T 149e740e79434d5794044d62740fc99f P 70e58bfbc17048e9a52605640790f5b4: Deleting consensus metadata
I20260619 20:46:56.139782 27885 catalog_manager.cc:4841] TS 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557): tablet 149e740e79434d5794044d62740fc99f (table test-table [id=92f156e90fd54873b5db29170ab043b1]) successfully deleted
W20260619 20:46:56.140540 27882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:34507 (error 108)
I20260619 20:46:56.140851 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 28094
I20260619 20:46:56.141054 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.140914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:34507 (local address 127.0.0.1:33388)
0619 20:46:56.140977 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.141015 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 20:46:56.142345 27882 catalog_manager.cc:4584] TS b48f0dc31d0c4887a36e875e1ed0ec06 (127.13.195.65:34507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 149e740e79434d5794044d62740fc99f: Network error: Client connection negotiation failed: client connection to 127.13.195.65:34507: connect: Connection refused (error 111)
I20260619 20:46:56.146011 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 28228
I20260619 20:46:56.150182 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 27866
I20260619 20:46:56.155287 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44785
--webserver_interface=127.13.195.126
--webserver_port=36611
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:56.236292 28399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:56.236529 28399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:56.236567 28399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:56.237749 28399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:56.237825 28399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:56.237850 28399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:56.237872 28399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:56.237900 28399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:56.239382 28399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:56.239655 28399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:56.239884 28399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:56.248389 28405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:56.248414 28406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:56.248411 28408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.248775 28399 server_base.cc:1034] running on GCE node
I20260619 20:46:56.249035 28399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:56.249366 28399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:56.250535 28399 hybrid_clock.cc:648] HybridClock initialized: now 1781902016250514 us; error 43 us; skew 500 ppm
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:56.253130 28399 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:46:56.253654 28399 webserver.cc:460] Webserver started at http://127.13.195.126:36611/ using document root <none> and password file <none>
I20260619 20:46:56.253911 28399 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:56.254009 28399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:56.255501 28399 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:56.256171 28415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.256350 28399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:46:56.256474 28399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "051d3ad8f83647c1a5b78f34f4b75b8c"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "76f89210f3eaacf6100debfd8698f072"
server_key_iv: "e48422d4ca3b4444e3ba4f62b05def11"
server_key_version: "encryptionkey@0"
I20260619 20:46:56.256901 28399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:56.273285 28399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:56.275296 28399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:56.275461 28399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:56.279577 28399 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:44785
I20260619 20:46:56.279696 28467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:44785 every 8 connection(s)
I20260619 20:46:56.280030 28399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:46:56.281599 28468 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:46:56.282294 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Bootstrap starting.
I20260619 20:46:56.289969 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28399
I20260619 20:46:56.290336 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:34507
--local_ip_for_outbound_sockets=127.13.195.65
--tserver_master_addrs=127.13.195.126:44785
--webserver_port=33831
--webserver_interface=127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:46:56.290889 28468 log.cc:824] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:56.292953 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:46:56.293278 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Bootstrap complete.
I20260619 20:46:56.295699 28468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.295948 28468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d3ad8f83647c1a5b78f34f4b75b8c, State: Initialized, Role: FOLLOWER
I20260619 20:46:56.296151 28468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.296257 28468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:46:56.296314 28468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:46:56.296362 28468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:46:56.297369 28468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.297488 28468 leader_election.cc:302] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 051d3ad8f83647c1a5b78f34f4b75b8c; no voters:
I20260619 20:46:56.297685 28468 leader_election.cc:288] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:46:56.297724 28472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:46:56.297825 28472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 LEADER]: Becoming Leader. State: Replica: 051d3ad8f83647c1a5b78f34f4b75b8c, State: Running, Role: LEADER
I20260619 20:46:56.297961 28468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:56.297941 28472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.298225 28472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 051d3ad8f83647c1a5b78f34f4b75b8c. Latest consensus state: current_term: 2 leader_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } } }
I20260619 20:46:56.298305 28472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:56.298233 28473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } } }
I20260619 20:46:56.298525 28473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:56.298534 28476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:46:56.299540 28476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:46:56.299687 28476 catalog_manager.cc:1259] Loaded cluster ID: 188e3420e7d84159844a80eaa6f7beac
I20260619 20:46:56.299811 28476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:46:56.300263 28476 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 20:46:56.304936 28476 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 20:46:56.307350 28476 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:46:56.307538 28476 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Loaded TSK: 0
I20260619 20:46:56.307767 28476 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:46:56.374197 28471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:56.374402 28471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:56.374444 28471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:56.375721 28471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:56.375792 28471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:56.375840 28471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:46:56.377322 28471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:34507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=33831
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:56.377602 28471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:56.377856 28471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:56.383113 28498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:56.383140 28495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:56.383113 28496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.383816 28471 server_base.cc:1034] running on GCE node
I20260619 20:46:56.384063 28471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:56.384328 28471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:56.385504 28471 hybrid_clock.cc:648] HybridClock initialized: now 1781902016385477 us; error 43 us; skew 500 ppm
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:56.388111 28471 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:46:56.388589 28471 webserver.cc:460] Webserver started at http://127.13.195.65:33831/ using document root <none> and password file <none>
I20260619 20:46:56.388841 28471 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:56.388901 28471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:56.390302 28471 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:56.391151 28505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.391346 28471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:56.391455 28471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "b48f0dc31d0c4887a36e875e1ed0ec06"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "9870a0de7ff65be70c1a648b8d9f2551"
server_key_iv: "ce2ac04b3e2610e526602751c36ef296"
server_key_version: "encryptionkey@0"
I20260619 20:46:56.391822 28471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:56.409011 28471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:56.411314 28471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:56.411492 28471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:56.411748 28471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:56.412277 28512 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:46:56.413779 28471 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:46:56.413834 28471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:46:56.413861 28471 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:46:56.414619 28471 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:46:56.414664 28471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:56.414713 28512 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Bootstrap starting.
I20260619 20:46:56.416808 28512 log.cc:824] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:56.420258 28512 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:46:56.420611 28512 tablet_bootstrap.cc:492] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Bootstrap complete.
I20260619 20:46:56.420794 28512 ts_tablet_manager.cc:1397] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260619 20:46:56.421864 28471 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:34507
I20260619 20:46:56.422250 28471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:46:56.423451 28512 raft_consensus.cc:348] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:56.423728 28512 raft_consensus.cc:729] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48f0dc31d0c4887a36e875e1ed0ec06, State: Initialized, Role: FOLLOWER
I20260619 20:46:56.423890 28512 consensus_queue.cc:260] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } }
I20260619 20:46:56.424154 28512 ts_tablet_manager.cc:1428] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:46:56.424393 28620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:34507 every 8 connection(s)
I20260619 20:46:56.425674 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28471
I20260619 20:46:56.425957 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:45557
--local_ip_for_outbound_sockets=127.13.195.66
--tserver_master_addrs=127.13.195.126:44785
--webserver_port=40225
--webserver_interface=127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:56.435844 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.423040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:37397 (local address 127.13.195.126:44785)
0619 20:46:56.424760 (+ 1720us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.424768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.424798 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:46:56.424902 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.424907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.424947 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.425055 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:56.425940 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.427290 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.428622 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.429008 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.429158 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.432397 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:56.432401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:56.432401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:56.432411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:56.434012 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.434554 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.434557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.434557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.434641 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.434950 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.434951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.434951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.435155 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:56.435217 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.435579 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.435697 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1671,"thread_start_us":72,"threads_started":1}
I20260619 20:46:56.435997 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.422913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:37397)
0619 20:46:56.423103 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.423146 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.423466 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.425130 (+ 1664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.425134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.425492 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:56.425811 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.425818 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.427421 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.427423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.428538 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.428541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.429134 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.429135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.429278 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.429870 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:56.429881 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:56.432277 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:56.434100 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.434107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.434109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.434471 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.434733 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.434734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.434734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.434878 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.435229 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:56.435238 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:56.435411 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.435538 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.435732 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260619 20:46:56.436794 28621 heartbeater.cc:347] Connected to a master server at 127.13.195.126:44785
I20260619 20:46:56.436908 28621 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:56.437202 28621 heartbeater.cc:511] Master 127.13.195.126:44785 requested a full tablet report, sending...
I20260619 20:46:56.438023 28432 ts_manager.cc:194] Registered new tserver with Master: b48f0dc31d0c4887a36e875e1ed0ec06 (127.13.195.65:34507)
I20260619 20:46:56.439214 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:37397
I20260619 20:46:56.440115 28621 heartbeater.cc:503] Master 127.13.195.126:44785 was elected leader, sending a full tablet report...
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:46:56.446781 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.438631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:34507 (local address 127.0.0.1:33402)
0619 20:46:56.438809 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.438849 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.438914 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.439100 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.439105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.439435 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:56.439491 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.439493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.440814 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.440815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.441364 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.441366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.441977 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.441978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.442032 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.442515 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:56.442520 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:56.443799 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:56.445755 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.445760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.445761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.445941 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.446152 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.446153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.446153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.446204 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.446504 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:56.446510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:56.446549 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.446610 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.446712 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260619 20:46:56.446846 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.438828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.13.195.65:34507)
0619 20:46:56.438959 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.438963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.438973 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:46:56.439003 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.439006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.439018 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.439087 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:56.439581 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.440735 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.441441 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.441903 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.441971 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.443879 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:56.443887 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:56.443887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:56.443894 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:56.445676 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.446023 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.446026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.446026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.446086 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.446261 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.446262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.446263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.446425 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:56.446528 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.446647 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.446740 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260619 20:46:56.447281 28555 tablet_service.cc:1514] Processing DeleteTablet for tablet 149e740e79434d5794044d62740fc99f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:56 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:33402
I20260619 20:46:56.447930 28629 tablet_replica.cc:330] stopping tablet replica
I20260619 20:46:56.448040 28629 raft_consensus.cc:2229] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:46:56.448132 28629 raft_consensus.cc:2258] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:56.448472 28629 ts_tablet_manager.cc:1905] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:56.450405 28629 ts_tablet_manager.cc:1918] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:56.450533 28629 log.cc:1196] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/149e740e79434d5794044d62740fc99f
I20260619 20:46:56.450769 28629 ts_tablet_manager.cc:1939] T 149e740e79434d5794044d62740fc99f P b48f0dc31d0c4887a36e875e1ed0ec06: Deleting consensus metadata
I20260619 20:46:56.451269 28416 catalog_manager.cc:4841] TS b48f0dc31d0c4887a36e875e1ed0ec06 (127.13.195.65:34507): tablet 149e740e79434d5794044d62740fc99f (table test-table [id=92f156e90fd54873b5db29170ab043b1]) successfully deleted
W20260619 20:46:56.513340 28626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:56.513587 28626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:56.513623 28626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:56.514810 28626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:56.514881 28626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:56.514926 28626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:46:56.516400 28626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:45557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=40225
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:56.516680 28626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:56.516924 28626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:56.521888 28637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.522073 28626 server_base.cc:1034] running on GCE node
W20260619 20:46:56.521936 28634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:56.521903 28635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.522401 28626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:56.522651 28626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:56.523794 28626 hybrid_clock.cc:648] HybridClock initialized: now 1781902016523774 us; error 40 us; skew 500 ppm
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:56.526319 28626 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:46:56.526774 28626 webserver.cc:460] Webserver started at http://127.13.195.66:40225/ using document root <none> and password file <none>
I20260619 20:46:56.527011 28626 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:56.527096 28626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:56.528478 28626 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:56.529156 28644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.529352 28626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:56.529435 28626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "70e58bfbc17048e9a52605640790f5b4"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "7df8352ff37ce336fce5c9e497f4893b"
server_key_iv: "eb3eceabad2beb53fbc849c11db27bf3"
server_key_version: "encryptionkey@0"
I20260619 20:46:56.529755 28626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:56.553066 28626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:56.555143 28626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:56.555258 28626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:56.555449 28626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:56.555789 28626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:56.555823 28626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.555847 28626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:56.555895 28626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.561568 28626 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:45557
I20260619 20:46:56.561671 28757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:45557 every 8 connection(s)
I20260619 20:46:56.562018 28626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:56.571103 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.562863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:60517 (local address 127.13.195.126:44785)
0619 20:46:56.562911 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.562913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.563157 (+ 244us) server_negotiation.cc:407] Connection header received
0619 20:46:56.563630 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.563630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.563644 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.563705 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:56.564625 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.565124 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.566060 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.566254 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.566304 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.568466 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:56.568468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:56.568468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:56.568475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:56.569901 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.570248 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.570248 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.570249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.570297 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.570548 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.570548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.570549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.570653 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:56.570720 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.570964 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.571035 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:46:56.571266 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.562873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:60517)
0619 20:46:56.563038 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.563075 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.563547 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.563779 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.563782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.564355 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:56.564548 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.564552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.565203 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.565204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.565984 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.565986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.566326 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.566327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.566488 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.566951 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:56.566959 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:56.568393 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:56.569965 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.569971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.569973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.570182 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.570354 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.570355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.570355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.570502 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.570727 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:56.570732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:56.570853 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.570937 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.571055 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260619 20:46:56.571727 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28626
I20260619 20:46:56.571892 28758 heartbeater.cc:347] Connected to a master server at 127.13.195.126:44785
I20260619 20:46:56.572000 28758 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:56.572050 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:35063
--local_ip_for_outbound_sockets=127.13.195.67
--tserver_master_addrs=127.13.195.126:44785
--webserver_port=39623
--webserver_interface=127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:46:56.572223 28758 heartbeater.cc:511] Master 127.13.195.126:44785 requested a full tablet report, sending...
I20260619 20:46:56.572645 28432 ts_manager.cc:194] Registered new tserver with Master: 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557)
I20260619 20:46:56.573035 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:60517
W20260619 20:46:56.653811 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:56.654021 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:56.654044 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:56.655304 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:56.655346 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:56.655373 28761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:46:56.656860 28761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:35063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=39623
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:56.657104 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:56.657333 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:56.662537 28767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.662600 28761 server_base.cc:1034] running on GCE node
W20260619 20:46:56.662600 28769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:56.662549 28766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.662894 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:56.663131 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:56.664285 28761 hybrid_clock.cc:648] HybridClock initialized: now 1781902016664285 us; error 37 us; skew 500 ppm
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:56.666767 28761 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:46:56.667200 28761 webserver.cc:460] Webserver started at http://127.13.195.67:39623/ using document root <none> and password file <none>
I20260619 20:46:56.667374 28761 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:56.667423 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:56.668843 28761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:56.669380 28776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.669598 28761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:46:56.669682 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "0a7d0ef9783a70113ffb7c212fee21ca"
server_key_iv: "12cbaf6dad199f847b5c5fd7cb28075b"
server_key_version: "encryptionkey@0"
I20260619 20:46:56.669996 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:56.692992 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:56.695206 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:56.695363 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:56.695567 28761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:56.695902 28761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:56.695940 28761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.695961 28761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:56.696008 28761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.702347 28761 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:35063
I20260619 20:46:56.702399 28889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:35063 every 8 connection(s)
I20260619 20:46:56.702805 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:46:56.707379 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28761
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:46:56.713012 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.703627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:45623 (local address 127.13.195.126:44785)
0619 20:46:56.703677 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.703679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.703897 (+ 218us) server_negotiation.cc:407] Connection header received
0619 20:46:56.704444 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.704446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.704458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.704511 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:56.705538 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.706068 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.707087 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.707294 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.707381 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.710047 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:56.710048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:56.710049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:56.710055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:56.711552 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.712012 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.712013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.712014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.712106 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.712352 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.712353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.712354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.712467 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:56.712543 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.712842 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.712921 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:46:56.713255 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.703594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:45623)
0619 20:46:56.703755 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.703791 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.704336 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.704594 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.704598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.705242 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:46:56.705438 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.705443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.706167 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.706168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.706993 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.706996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.707392 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.707393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.707559 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.708089 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:46:56.708097 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:46:56.709959 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:46:56.711632 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.711640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.711642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.711941 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.712172 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:46:56.712172 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:46:56.712173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:46:56.712276 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:46:56.712547 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:46:56.712553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:46:56.712689 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.712808 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.712964 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260619 20:46:56.713754 28890 heartbeater.cc:347] Connected to a master server at 127.13.195.126:44785
I20260619 20:46:56.713856 28890 heartbeater.cc:464] Registering TS with master...
I20260619 20:46:56.714057 28890 heartbeater.cc:511] Master 127.13.195.126:44785 requested a full tablet report, sending...
I20260619 20:46:56.714538 28432 ts_manager.cc:194] Registered new tserver with Master: 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063)
I20260619 20:46:56.714982 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:45623
I20260619 20:46:56.715065 28894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.707791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.13.195.126:44785)
0619 20:46:56.708061 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.708063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.708070 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:46:56.708087 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.708087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.708096 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.708155 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:46:56.708895 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.709642 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.710520 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.710732 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.710816 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.712018 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:46:56.712019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:46:56.712019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:46:56.712025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:46:56.713773 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.714138 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.714139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.714139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.714198 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:46:56.714512 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:46:56.714513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:46:56.714513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:46:56.714632 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:46:56.714824 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.714862 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.714938 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":71,"threads_started":1}
I20260619 20:46:56.715749 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:46:56.718183 28894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.716056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.13.195.126:44785)
0619 20:46:56.716091 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.716093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.716181 (+ 88us) server_negotiation.cc:407] Connection header received
0619 20:46:56.716237 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.716238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.716263 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.716321 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:46:56.716386 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.716816 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.717505 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.717698 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.717764 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.717880 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:46:56.717966 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:46:56.718023 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.718054 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.718072 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:46:56.718503 28432 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:46:56.720005 28432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:46:56.722210 28555 tablet_service.cc:1467] Processing CreateTablet for tablet 153fe9da5bf84d74a8e71d51b4f3e89d (DEFAULT_TABLE table=test-table [id=ec391e2cd0cd4dbf8d7507a2fd237f7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:56.722414 28555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153fe9da5bf84d74a8e71d51b4f3e89d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:56.724205 28512 tablet_bootstrap.cc:492] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: Bootstrap starting.
I20260619 20:46:56.724635 28512 tablet_bootstrap.cc:654] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:56.725312 28512 tablet_bootstrap.cc:492] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: No bootstrap required, opened a new log
I20260619 20:46:56.725368 28512 ts_tablet_manager.cc:1397] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:56.725670 28512 raft_consensus.cc:348] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.725754 28512 raft_consensus.cc:374] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:56.725804 28512 raft_consensus.cc:729] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48f0dc31d0c4887a36e875e1ed0ec06, State: Initialized, Role: FOLLOWER
I20260619 20:46:56.725850 28512 consensus_queue.cc:260] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.725943 28512 ts_tablet_manager.cc:1428] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:56.726507 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.721896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:35063 (local address 127.0.0.1:42154)
0619 20:46:56.722051 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.722064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.722105 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.723010 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.723011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.723013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:56.723079 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.723081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.724020 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.724021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.725229 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.725232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.726197 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.726198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.726275 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.726298 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.726346 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.726395 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260619 20:46:56.726604 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.721976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.13.195.67:35063)
0619 20:46:56.722805 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.722811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.722820 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:46:56.722863 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.722865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.722878 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.723160 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:56.723220 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.723913 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.725318 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.726108 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.726184 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.726241 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.726379 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.726419 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":113,"threads_started":1}
I20260619 20:46:56.726759 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.722020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:45557 (local address 127.0.0.1:36034)
0619 20:46:56.722623 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.722634 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.722689 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.723062 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.723063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.723065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:56.723107 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.723109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.724522 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.724523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.725515 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.725518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.726535 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.726536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.726599 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.726602 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.726653 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.726693 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":79,"threads_started":1}
I20260619 20:46:56.726795 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.722633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36034 (local address 127.13.195.66:45557)
0619 20:46:56.722834 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.722840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.722852 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:46:56.722891 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.722893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.722908 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.723176 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:56.723254 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.724414 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.725615 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.726337 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.726382 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.726473 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.726677 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.726717 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260619 20:46:56.727406 28824 tablet_service.cc:1467] Processing CreateTablet for tablet 153fe9da5bf84d74a8e71d51b4f3e89d (DEFAULT_TABLE table=test-table [id=ec391e2cd0cd4dbf8d7507a2fd237f7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:56.727504 28692 tablet_service.cc:1467] Processing CreateTablet for tablet 153fe9da5bf84d74a8e71d51b4f3e89d (DEFAULT_TABLE table=test-table [id=ec391e2cd0cd4dbf8d7507a2fd237f7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:46:56.727764 28692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153fe9da5bf84d74a8e71d51b4f3e89d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:56.727764 28824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153fe9da5bf84d74a8e71d51b4f3e89d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:46:56.731312 28901 tablet_bootstrap.cc:492] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Bootstrap starting.
I20260619 20:46:56.731977 28901 tablet_bootstrap.cc:654] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:56.732425 28901 log.cc:824] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:56.732825 28900 tablet_bootstrap.cc:492] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Bootstrap starting.
I20260619 20:46:56.733341 28900 tablet_bootstrap.cc:654] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Neither blocks nor log segments found. Creating new log.
I20260619 20:46:56.733362 28901 tablet_bootstrap.cc:492] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: No bootstrap required, opened a new log
I20260619 20:46:56.733459 28901 ts_tablet_manager.cc:1397] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:46:56.733909 28900 log.cc:824] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:56.734663 28900 tablet_bootstrap.cc:492] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: No bootstrap required, opened a new log
I20260619 20:46:56.734747 28900 ts_tablet_manager.cc:1397] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:46:56.735256 28901 raft_consensus.cc:348] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.735390 28901 raft_consensus.cc:374] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:56.735412 28901 raft_consensus.cc:729] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef3f492a39d457a8bcfaf9f7fd1a0ce, State: Initialized, Role: FOLLOWER
I20260619 20:46:56.735536 28901 consensus_queue.cc:260] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.735814 28901 ts_tablet_manager.cc:1428] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:46:56.735879 28890 heartbeater.cc:503] Master 127.13.195.126:44785 was elected leader, sending a full tablet report...
I20260619 20:46:56.736644 28900 raft_consensus.cc:348] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.736807 28900 raft_consensus.cc:374] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:46:56.736845 28900 raft_consensus.cc:729] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e58bfbc17048e9a52605640790f5b4, State: Initialized, Role: FOLLOWER
I20260619 20:46:56.736949 28900 consensus_queue.cc:260] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.737232 28900 ts_tablet_manager.cc:1428] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:46:56.737273 28758 heartbeater.cc:503] Master 127.13.195.126:44785 was elected leader, sending a full tablet report...
I20260619 20:46:56.763316 28625 raft_consensus.cc:482] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:46:56.763444 28625 raft_consensus.cc:504] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.763849 28625 leader_election.cc:288] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063), 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557)
I20260619 20:46:56.767030 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.764131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:56507 (local address 127.13.195.67:35063)
0619 20:46:56.764189 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.764191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.764202 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:46:56.764315 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.764316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.764326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.764400 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:56.764592 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.765115 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.766195 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.766650 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.766744 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.766753 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.766911 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.766954 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:46:56.767036 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.764266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:55409 (local address 127.13.195.66:45557)
0619 20:46:56.764316 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.764318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.764447 (+ 129us) server_negotiation.cc:407] Connection header received
0619 20:46:56.764480 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.764481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.764493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.764557 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:46:56.764713 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.765135 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.766201 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.766653 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.766743 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.766752 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.766936 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.766974 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:46:56.767036 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.764053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:35063 (local address 127.13.195.65:56507)
0619 20:46:56.764116 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.764128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.764185 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.764442 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.764443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.764446 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:56.764523 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.764525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.765199 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.765199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.766099 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.766102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.766743 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.766744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.766825 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.766842 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.766916 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.766960 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":57}
I20260619 20:46:56.767069 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.764184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:45557 (local address 127.13.195.65:55409)
0619 20:46:56.764359 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:56.764370 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:56.764438 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:56.764580 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:46:56.764581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:46:56.764582 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:46:56.764648 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.764649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.765270 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.765270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.766101 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:46:56.766103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.766743 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.766744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:46:56.766831 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.766846 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:46:56.766919 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:46:56.766956 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260619 20:46:56.767405 28712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153fe9da5bf84d74a8e71d51b4f3e89d" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e58bfbc17048e9a52605640790f5b4" is_pre_election: true
I20260619 20:46:56.767462 28844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153fe9da5bf84d74a8e71d51b4f3e89d" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" is_pre_election: true
I20260619 20:46:56.767554 28712 raft_consensus.cc:2454] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 0.
I20260619 20:46:56.767658 28844 raft_consensus.cc:2454] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 0.
I20260619 20:46:56.767831 28509 leader_election.cc:302] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e58bfbc17048e9a52605640790f5b4, b48f0dc31d0c4887a36e875e1ed0ec06; no voters:
I20260619 20:46:56.767947 28625 raft_consensus.cc:2781] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:46:56.768002 28625 raft_consensus.cc:482] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:46:56.768036 28625 raft_consensus.cc:3037] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:56.768800 28625 raft_consensus.cc:504] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.768949 28625 leader_election.cc:288] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 election: Requested vote from peers 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063), 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557)
I20260619 20:46:56.769052 28844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153fe9da5bf84d74a8e71d51b4f3e89d" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce"
I20260619 20:46:56.769084 28712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153fe9da5bf84d74a8e71d51b4f3e89d" candidate_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e58bfbc17048e9a52605640790f5b4"
I20260619 20:46:56.769146 28712 raft_consensus.cc:3037] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:56.769146 28844 raft_consensus.cc:3037] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:46:56.769937 28844 raft_consensus.cc:2454] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 1.
I20260619 20:46:56.769950 28712 raft_consensus.cc:2454] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b48f0dc31d0c4887a36e875e1ed0ec06 in term 1.
I20260619 20:46:56.770087 28509 leader_election.cc:302] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ef3f492a39d457a8bcfaf9f7fd1a0ce, b48f0dc31d0c4887a36e875e1ed0ec06; no voters:
I20260619 20:46:56.770164 28625 raft_consensus.cc:2781] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:46:56.770298 28625 raft_consensus.cc:686] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 LEADER]: Becoming Leader. State: Replica: b48f0dc31d0c4887a36e875e1ed0ec06, State: Running, Role: LEADER
I20260619 20:46:56.770355 28625 consensus_queue.cc:237] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } }
I20260619 20:46:56.771178 28432 catalog_manager.cc:5495] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 reported cstate change: term changed from 0 to 1, leader changed from <none> to b48f0dc31d0c4887a36e875e1ed0ec06 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b48f0dc31d0c4887a36e875e1ed0ec06" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 34507 } health_report { overall_health: HEALTHY } } }
I20260619 20:46:56.784888 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:56.782365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.13.195.65:34507)
0619 20:46:56.782448 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:46:56.782450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:46:56.782459 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:46:56.782542 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:46:56.782543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:46:56.782566 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:46:56.782639 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:46:56.782827 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.783315 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.784067 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:46:56.784288 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:46:56.784371 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:46:56.784550 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:46:56.784650 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:46:56.784734 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 20:46:56.784773 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 20:46:56.784793 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:46:56.786449 28844 raft_consensus.cc:1261] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Refusing update from remote peer b48f0dc31d0c4887a36e875e1ed0ec06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:56.786453 28712 raft_consensus.cc:1261] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Refusing update from remote peer b48f0dc31d0c4887a36e875e1ed0ec06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:46:56.786801 28625 consensus_queue.cc:1035] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e58bfbc17048e9a52605640790f5b4" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 45557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:56.786962 28907 consensus_queue.cc:1035] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:46:56.789222 28909 mvcc.cc:205] Tried to move back new op lower bound from 7298670660754640896 to 7298670660692049920. Current Snapshot: MvccSnapshot[applied={T|T < 7298670660754640896}]
I20260619 20:46:56.790176 28910 mvcc.cc:205] Tried to move back new op lower bound from 7298670660754640896 to 7298670660692049920. Current Snapshot: MvccSnapshot[applied={T|T < 7298670660754640896}]
I20260619 20:46:56.791999 28432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:56.792093 28432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:46:56.792752 28432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Sending DeleteTablet for 3 replicas of tablet 153fe9da5bf84d74a8e71d51b4f3e89d
I20260619 20:46:56.793044 28555 tablet_service.cc:1514] Processing DeleteTablet for tablet 153fe9da5bf84d74a8e71d51b4f3e89d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:56 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:33402
I20260619 20:46:56.793071 28824 tablet_service.cc:1514] Processing DeleteTablet for tablet 153fe9da5bf84d74a8e71d51b4f3e89d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:56 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:42154
I20260619 20:46:56.793097 28692 tablet_service.cc:1514] Processing DeleteTablet for tablet 153fe9da5bf84d74a8e71d51b4f3e89d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:46:56 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:36034
I20260619 20:46:56.793136 28629 tablet_replica.cc:330] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: stopping tablet replica
I20260619 20:46:56.793213 28629 raft_consensus.cc:2229] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:46:56.793280 28920 tablet_replica.cc:330] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: stopping tablet replica
I20260619 20:46:56.793298 28629 raft_consensus.cc:2258] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:56.793305 28921 tablet_replica.cc:330] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: stopping tablet replica
I20260619 20:46:56.793375 28920 raft_consensus.cc:2229] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:46:56.793386 28921 raft_consensus.cc:2229] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:46:56.793473 28921 raft_consensus.cc:2258] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:56.793474 28920 raft_consensus.cc:2258] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:46:56.793748 28629 ts_tablet_manager.cc:1905] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:56.793884 28920 ts_tablet_manager.cc:1905] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:56.793888 28921 ts_tablet_manager.cc:1905] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:46:56.795342 28629 ts_tablet_manager.cc:1918] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:56.795392 28629 log.cc:1196] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/153fe9da5bf84d74a8e71d51b4f3e89d
I20260619 20:46:56.795519 28921 ts_tablet_manager.cc:1918] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:56.795523 28920 ts_tablet_manager.cc:1918] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:46:56.795583 28921 log.cc:1196] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/153fe9da5bf84d74a8e71d51b4f3e89d
I20260619 20:46:56.795585 28920 log.cc:1196] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/153fe9da5bf84d74a8e71d51b4f3e89d
I20260619 20:46:56.795625 28629 ts_tablet_manager.cc:1939] T 153fe9da5bf84d74a8e71d51b4f3e89d P b48f0dc31d0c4887a36e875e1ed0ec06: Deleting consensus metadata
I20260619 20:46:56.795876 28920 ts_tablet_manager.cc:1939] T 153fe9da5bf84d74a8e71d51b4f3e89d P 7ef3f492a39d457a8bcfaf9f7fd1a0ce: Deleting consensus metadata
I20260619 20:46:56.795969 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 28471
I20260619 20:46:56.796552 28419 catalog_manager.cc:4841] TS 7ef3f492a39d457a8bcfaf9f7fd1a0ce (127.13.195.67:35063): tablet 153fe9da5bf84d74a8e71d51b4f3e89d (table test-table [id=ec391e2cd0cd4dbf8d7507a2fd237f7f]) successfully deleted
I20260619 20:46:56.796614 28921 ts_tablet_manager.cc:1939] T 153fe9da5bf84d74a8e71d51b4f3e89d P 70e58bfbc17048e9a52605640790f5b4: Deleting consensus metadata
I20260619 20:46:56.796973 28419 catalog_manager.cc:4841] TS 70e58bfbc17048e9a52605640790f5b4 (127.13.195.66:45557): tablet 153fe9da5bf84d74a8e71d51b4f3e89d (table test-table [id=ec391e2cd0cd4dbf8d7507a2fd237f7f]) successfully deleted
I20260619 20:46:56.798702 28416 catalog_manager.cc:4841] TS b48f0dc31d0c4887a36e875e1ed0ec06 (127.13.195.65:34507): tablet 153fe9da5bf84d74a8e71d51b4f3e89d (table test-table [id=ec391e2cd0cd4dbf8d7507a2fd237f7f]) successfully deleted
I20260619 20:46:56.802392 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 28626
I20260619 20:46:56.807070 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 28761
I20260619 20:46:56.811637 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 28399
I20260619 20:46:56.816934 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44785
--webserver_interface=127.13.195.126
--webserver_port=36611
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:56.897647 28922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:56.897879 28922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:56.897902 28922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:56.899091 28922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:46:56.899132 28922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:56.899147 28922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:56.899159 28922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:46:56.899217 28922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:46:56.900641 28922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=36611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:56.900900 28922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:56.901139 28922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:56.905963 28930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.906117 28922 server_base.cc:1034] running on GCE node
W20260619 20:46:56.906091 28928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:56.905975 28927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:56.906569 28922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:56.906801 28922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:56.907934 28922 hybrid_clock.cc:648] HybridClock initialized: now 1781902016907924 us; error 26 us; skew 500 ppm
Jun 19 20:46:56 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902016, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:56.910395 28922 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:46:56.910823 28922 webserver.cc:460] Webserver started at http://127.13.195.126:36611/ using document root <none> and password file <none>
I20260619 20:46:56.911018 28922 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:56.911059 28922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:56.912356 28922 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:56.912976 28937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:56.913134 28922 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:46:56.913210 28922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "051d3ad8f83647c1a5b78f34f4b75b8c"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "76f89210f3eaacf6100debfd8698f072"
server_key_iv: "e48422d4ca3b4444e3ba4f62b05def11"
server_key_version: "encryptionkey@0"
I20260619 20:46:56.913513 28922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:56.945776 28922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:56.948196 28922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:56.948324 28922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:56.952306 28922 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:44785
I20260619 20:46:56.952348 28989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:44785 every 8 connection(s)
I20260619 20:46:56.952735 28922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:46:56.954277 28990 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:46:56.954964 28990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Bootstrap starting.
I20260619 20:46:56.961937 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28922
I20260619 20:46:56.962244 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:34507
--local_ip_for_outbound_sockets=127.13.195.65
--tserver_master_addrs=127.13.195.126:44785
--webserver_port=33831
--webserver_interface=127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:46:56.963451 28990 log.cc:824] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:46:56.965750 28990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:46:56.966037 28990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c: Bootstrap complete.
I20260619 20:46:56.968462 28990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.968721 28990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051d3ad8f83647c1a5b78f34f4b75b8c, State: Initialized, Role: FOLLOWER
I20260619 20:46:56.968916 28990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.969012 28990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:46:56.969064 28990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:46:56.969112 28990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:46:56.970202 28990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.970321 28990 leader_election.cc:302] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 051d3ad8f83647c1a5b78f34f4b75b8c; no voters:
I20260619 20:46:56.970512 28990 leader_election.cc:288] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 20:46:56.970554 28994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 3 FOLLOWER]: Leader election won for term 3
I20260619 20:46:56.970661 28994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [term 3 LEADER]: Becoming Leader. State: Replica: 051d3ad8f83647c1a5b78f34f4b75b8c, State: Running, Role: LEADER
I20260619 20:46:56.970755 28994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } }
I20260619 20:46:56.970791 28990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:46:56.971004 28995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } } }
I20260619 20:46:56.971107 28995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:56.971293 28996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 051d3ad8f83647c1a5b78f34f4b75b8c. Latest consensus state: current_term: 3 leader_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051d3ad8f83647c1a5b78f34f4b75b8c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44785 } } }
I20260619 20:46:56.971370 28996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051d3ad8f83647c1a5b78f34f4b75b8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:46:56.971336 28998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:46:56.972450 28998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:46:56.972621 28998 catalog_manager.cc:1259] Loaded cluster ID: 188e3420e7d84159844a80eaa6f7beac
I20260619 20:46:56.972784 28998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 20:46:56.973006 28998 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3985a659ad google::LogMessage::Fail() at ??:0
@ 0x7f3985a6888a google::LogMessage::SendToLog() at ??:0
@ 0x7f3985a654de google::LogMessage::Flush() at ??:0
@ 0x7f3985a66a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f398d9860f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f398d98f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f39879330d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3987928954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f398878c6db start_thread at ??:0
@ 0x7f398456361f clone at ??:0
W20260619 20:46:57.060662 28992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:57.060860 28992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:57.060884 28992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:57.062036 28992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:57.062076 28992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:57.062103 28992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:46:57.063606 28992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:34507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=33831
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:57.063861 28992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:57.064101 28992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:57.069502 29026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:57.069545 29029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:57.069617 29027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:57.070144 28992 server_base.cc:1034] running on GCE node
I20260619 20:46:57.070379 28992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:57.070672 28992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:57.071828 28992 hybrid_clock.cc:648] HybridClock initialized: now 1781902017071818 us; error 25 us; skew 500 ppm
Jun 19 20:46:57 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902017, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:57.074499 28992 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:46:57.074985 28992 webserver.cc:460] Webserver started at http://127.13.195.65:33831/ using document root <none> and password file <none>
I20260619 20:46:57.075244 28992 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:57.075309 28992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:57.076742 28992 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:57.077575 29036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.077744 28992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:46:57.077831 28992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "b48f0dc31d0c4887a36e875e1ed0ec06"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "9870a0de7ff65be70c1a648b8d9f2551"
server_key_iv: "ce2ac04b3e2610e526602751c36ef296"
server_key_version: "encryptionkey@0"
I20260619 20:46:57.078337 28992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:57.107611 28992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:57.109884 28992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:57.110029 28992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:57.110280 28992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:57.110656 28992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:57.110713 28992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.110749 28992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:57.110775 28992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.116838 28992 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:34507
I20260619 20:46:57.116887 29149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:34507 every 8 connection(s)
I20260619 20:46:57.117270 28992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:46:57.118599 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 28992
I20260619 20:46:57.118888 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:45557
--local_ip_for_outbound_sockets=127.13.195.66
--tserver_master_addrs=127.13.195.126:44785
--webserver_port=40225
--webserver_interface=127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:57.180452 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:46:57.118044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:45329)
0619 20:46:57.118221 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.118274 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 20:46:57.119014 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:46:57.179927 (+ 60913us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
W20260619 20:46:57.180740 29150 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:44785 (0 consecutive failures): Network error: Failed to ping master at 127.13.195.126:44785: Client connection negotiation failed: client connection to 127.13.195.126:44785: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:46:57.181080 29152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.180959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:56187)
0619 20:46:57.181013 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.181059 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:46:57.181448 29152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.181321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:55591)
0619 20:46:57.181374 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.181418 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:46:57.181629 29150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:46:57.203452 29153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:57.203675 29153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:57.203698 29153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:57.204878 29153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:57.204916 29153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:57.204942 29153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:46:57.206354 29153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:45557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=40225
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:57.206612 29153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:57.206864 29153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:57.211997 29158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:57.212013 29161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:57.212013 29159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:57.212270 29153 server_base.cc:1034] running on GCE node
I20260619 20:46:57.212448 29153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:57.212683 29153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:57.213825 29153 hybrid_clock.cc:648] HybridClock initialized: now 1781902017213813 us; error 30 us; skew 500 ppm
Jun 19 20:46:57 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902017, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:57.216311 29153 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:46:57.216769 29153 webserver.cc:460] Webserver started at http://127.13.195.66:40225/ using document root <none> and password file <none>
I20260619 20:46:57.217002 29153 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:57.217047 29153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:57.218418 29153 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:46:57.219157 29168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.219393 29153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:57.219501 29153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "70e58bfbc17048e9a52605640790f5b4"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "7df8352ff37ce336fce5c9e497f4893b"
server_key_iv: "eb3eceabad2beb53fbc849c11db27bf3"
server_key_version: "encryptionkey@0"
I20260619 20:46:57.219877 29153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:57.231123 29153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:57.233125 29153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:57.233286 29153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:57.233521 29153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:57.233898 29153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:57.233960 29153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.234004 29153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:57.234030 29153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.240087 29153 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:45557
I20260619 20:46:57.240137 29281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:45557 every 8 connection(s)
I20260619 20:46:57.240480 29153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:46:57.241590 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.241122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:40497)
0619 20:46:57.241272 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.241418 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
W20260619 20:46:57.242065 29282 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:44785 (0 consecutive failures): Network error: Failed to ping master at 127.13.195.126:44785: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
I20260619 20:46:57.242512 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.242371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:37751)
0619 20:46:57.242419 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.242485 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:46:57.242892 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.242780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:36973)
0619 20:46:57.242845 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.242872 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 20:46:57.243029 29282 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:46:57.244071 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 29153
I20260619 20:46:57.244328 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:35063
--local_ip_for_outbound_sockets=127.13.195.67
--tserver_master_addrs=127.13.195.126:44785
--webserver_port=39623
--webserver_interface=127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:46:57.326864 29285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:46:57.327065 29285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:46:57.327088 29285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:46:57.328249 29285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:46:57.328312 29285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:46:57.328339 29285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:46:57.329823 29285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:35063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=39623
--tserver_master_addrs=127.13.195.126:44785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:46:57.330073 29285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:46:57.330317 29285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:46:57.335438 29290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:57.335419 29293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:46:57.335407 29291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:46:57.335605 29285 server_base.cc:1034] running on GCE node
I20260619 20:46:57.335914 29285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:46:57.336184 29285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:46:57.337353 29285 hybrid_clock.cc:648] HybridClock initialized: now 1781902017337340 us; error 28 us; skew 500 ppm
Jun 19 20:46:57 dist-test-slave-n2td krb5kdc[27853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902017, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:46:57.339982 29285 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:46:57.340379 29285 webserver.cc:460] Webserver started at http://127.13.195.67:39623/ using document root <none> and password file <none>
I20260619 20:46:57.340576 29285 fs_manager.cc:359] Metadata directory not provided
I20260619 20:46:57.340617 29285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:46:57.341913 29285 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:57.342586 29300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.342756 29285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:46:57.342855 29285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "7ef3f492a39d457a8bcfaf9f7fd1a0ce"
format_stamp: "Formatted at 2026-06-19 20:46:55 on dist-test-slave-n2td"
server_key: "0a7d0ef9783a70113ffb7c212fee21ca"
server_key_iv: "12cbaf6dad199f847b5c5fd7cb28075b"
server_key_version: "encryptionkey@0"
I20260619 20:46:57.343202 29285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:46:57.355960 29285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:46:57.358251 29285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:46:57.358364 29285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:46:57.358655 29285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:46:57.358981 29285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:46:57.359043 29285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.359086 29285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:46:57.359112 29285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:46:57.364907 29285 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:35063
I20260619 20:46:57.364950 29413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:35063 every 8 connection(s)
I20260619 20:46:57.365329 29285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:46:57.366405 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.366026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:49263)
0619 20:46:57.366185 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.366263 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
W20260619 20:46:57.366878 29414 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:44785 (0 consecutive failures): Network error: Failed to ping master at 127.13.195.126:44785: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
I20260619 20:46:57.367180 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.367054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:43267)
0619 20:46:57.367126 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.367158 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 20:46:57.367504 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:57.367426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:46935)
0619 20:46:57.367463 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:57.367486 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 20:46:57.367650 29414 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:46:57.369032 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 29285
I20260619 20:46:58.182157 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:58.181949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:58943)
0619 20:46:58.182060 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:58.182093 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":78,"threads_started":1}
I20260619 20:46:58.243616 29419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:58.243387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:42973)
0619 20:46:58.243520 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:58.243561 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
I20260619 20:46:58.368208 29420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:58.367987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:59177)
0619 20:46:58.368114 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:58.368159 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260619 20:46:59.182907 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:59.182664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:46779)
0619 20:46:59.182818 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:59.182861 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260619 20:46:59.244357 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:59.244145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:55559)
0619 20:46:59.244277 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:59.244316 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260619 20:46:59.369128 29423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:46:59.368806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:45113)
0619 20:46:59.369002 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 20:46:59.369074 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260619 20:47:00.183585 29424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:00.183369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:58001)
0619 20:47:00.183501 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:00.183538 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260619 20:47:00.245155 29425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:00.244910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:60797)
0619 20:47:00.245066 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:00.245110 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260619 20:47:00.369827 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:00.369602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:52975)
0619 20:47:00.369745 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:00.369782 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260619 20:47:01.184316 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:01.184091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:39411)
0619 20:47:01.184219 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:01.184267 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260619 20:47:01.245921 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:01.245664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:33127)
0619 20:47:01.245796 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:01.245835 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
I20260619 20:47:01.370568 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:01.370271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:36811)
0619 20:47:01.370406 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:01.370520 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260619 20:47:02.185030 29430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:02.184807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:48149)
0619 20:47:02.184943 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:02.184981 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260619 20:47:02.246690 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:02.246424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:46617)
0619 20:47:02.246605 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:02.246645 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260619 20:47:02.371266 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:02.371032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:53895)
0619 20:47:02.371171 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:02.371210 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260619 20:47:03.185861 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:03.185547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:43777)
0619 20:47:03.185727 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:03.185808 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":65,"threads_started":1}
I20260619 20:47:03.247438 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:03.247216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:53029)
0619 20:47:03.247348 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:03.247391 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260619 20:47:03.372107 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:03.371824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:39137)
0619 20:47:03.371996 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:03.372038 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260619 20:47:04.186661 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:04.186394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:40699)
0619 20:47:04.186574 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:04.186613 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260619 20:47:04.248132 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:04.247903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:39061)
0619 20:47:04.248036 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:04.248072 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260619 20:47:04.372828 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:04.372575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:33733)
0619 20:47:04.372730 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:04.372773 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260619 20:47:05.187582 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:05.187292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:37169)
0619 20:47:05.187457 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:05.187516 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260619 20:47:05.248931 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:05.248681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:57987)
0619 20:47:05.248835 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:05.248883 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260619 20:47:05.373582 29441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:05.373330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:37807)
0619 20:47:05.373469 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:05.373510 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260619 20:47:06.188444 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:06.188138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:35393)
0619 20:47:06.188337 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:06.188394 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
I20260619 20:47:06.249644 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:06.249411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:55589)
0619 20:47:06.249561 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:06.249599 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260619 20:47:06.374468 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:06.374164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:55223)
0619 20:47:06.374340 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:06.374387 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":75,"threads_started":1}
I20260619 20:47:07.189311 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:07.189066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:59493)
0619 20:47:07.189213 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:07.189261 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260619 20:47:07.250486 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:07.250195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:40085)
0619 20:47:07.250358 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:07.250407 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260619 20:47:07.375392 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:07.375116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:42897)
0619 20:47:07.375290 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:07.375342 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
I20260619 20:47:08.190054 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:08.189799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:47995)
0619 20:47:08.189959 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:08.190004 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":64,"threads_started":1}
I20260619 20:47:08.251328 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:08.251083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:34883)
0619 20:47:08.251235 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:08.251278 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":57,"threads_started":1}
I20260619 20:47:08.376149 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:08.375860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:35549)
0619 20:47:08.376036 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:08.376079 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260619 20:47:09.190805 29451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:09.190605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:45789)
0619 20:47:09.190721 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:09.190757 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260619 20:47:09.252067 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:09.251832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:34259)
0619 20:47:09.251983 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:09.252022 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":73,"threads_started":1}
I20260619 20:47:09.377146 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:09.376901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:36621)
0619 20:47:09.377056 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:09.377098 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260619 20:47:10.191569 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:10.191347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:36199)
0619 20:47:10.191478 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:10.191521 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260619 20:47:10.252844 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:10.252599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:58879)
0619 20:47:10.252751 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:10.252796 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260619 20:47:10.377869 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:10.377637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:58349)
0619 20:47:10.377781 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:10.377822 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260619 20:47:11.192322 29457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:11.192066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:43879)
0619 20:47:11.192220 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:11.192271 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260619 20:47:11.253562 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:11.253346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:35475)
0619 20:47:11.253474 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:11.253513 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260619 20:47:11.378592 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:11.378339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:53595)
0619 20:47:11.378507 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:11.378545 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260619 20:47:12.193316 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:12.192946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.65:49975)
0619 20:47:12.193222 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:12.193266 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":74,"threads_started":1}
I20260619 20:47:12.254276 29461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:12.254069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.66:50475)
0619 20:47:12.254193 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:12.254230 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260619 20:47:12.374866 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 28992
I20260619 20:47:12.379312 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:12.379104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:44785 (local address 127.13.195.67:36333)
0619 20:47:12.379228 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:12.379269 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260619 20:47:12.379726 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 29153
I20260619 20:47:12.383983 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 29285
2026-06-19T20:47:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18540 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:47:12 dist-test-slave-n2td krb5kdc[29465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:47:12 dist-test-slave-n2td krb5kdc[29465](info): set up 2 sockets
Jun 19 20:47:12 dist-test-slave-n2td krb5kdc[29465](info): commencing operation
krb5kdc: starting...
W20260619 20:47:13.441556 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:47:13 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:47:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:47:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:47:13.562176 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45713
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:47:13.649363 29478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:13.649601 29478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:13.649638 29478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:13.650858 29478 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:47:13.650915 29478 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:47:13.650936 29478 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:47:13.650966 29478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:47:13.650992 29478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:13.651031 29478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:13.651060 29478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:47:13.651086 29478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:47:13.652552 29478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:13.652830 29478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:13.653059 29478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:13.658285 29484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:13.658252 29486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:13.658371 29478 server_base.cc:1034] running on GCE node
W20260619 20:47:13.658262 29483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:13.658713 29478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:13.659008 29478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:13.660182 29478 hybrid_clock.cc:648] HybridClock initialized: now 1781902033660174 us; error 40 us; skew 500 ppm
Jun 19 20:47:13 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902033, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:13.662645 29478 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:47:13.663132 29478 webserver.cc:460] Webserver started at http://127.13.195.126:43699/ using document root <none> and password file <none>
I20260619 20:47:13.663357 29478 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:13.663424 29478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:13.663578 29478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:47:13.664670 29478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "9ff15424a9d54e6fbb43bde3ab200236"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "a89d76965ad7fe1aa84a6067c44d9f24"
server_key_iv: "fcd0d8d1ab91faf417d50a10cf99c74f"
server_key_version: "encryptionkey@0"
I20260619 20:47:13.665040 29478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "9ff15424a9d54e6fbb43bde3ab200236"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "a89d76965ad7fe1aa84a6067c44d9f24"
server_key_iv: "fcd0d8d1ab91faf417d50a10cf99c74f"
server_key_version: "encryptionkey@0"
I20260619 20:47:13.666364 29478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:47:13.667161 29493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:13.667388 29478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:13.667510 29478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "9ff15424a9d54e6fbb43bde3ab200236"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "a89d76965ad7fe1aa84a6067c44d9f24"
server_key_iv: "fcd0d8d1ab91faf417d50a10cf99c74f"
server_key_version: "encryptionkey@0"
I20260619 20:47:13.667619 29478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:13.700285 29478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:13.702255 29478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:13.702451 29478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:13.706382 29478 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:45713
I20260619 20:47:13.706416 29545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:45713 every 8 connection(s)
I20260619 20:47:13.706828 29478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:47:13.707706 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:47:13.707898 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 29478
I20260619 20:47:13.708010 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:47:13.708213 14093 external_mini_cluster.cc:1428] Setting key 82b75cbc70fdd43082604a4dee67b50e
I20260619 20:47:13.710763 29546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Bootstrap starting.
I20260619 20:47:13.711386 29546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Neither blocks nor log segments found. Creating new log.
I20260619 20:47:13.711673 29546 log.cc:824] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:13.712472 29546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: No bootstrap required, opened a new log
Jun 19 20:47:13 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:47:13.714855 29546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:13.715030 29546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:47:13.715077 29546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff15424a9d54e6fbb43bde3ab200236, State: Initialized, Role: FOLLOWER
I20260619 20:47:13.715205 29546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:13.715308 29546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:47:13.715343 29546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:47:13.715397 29546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:47:13.716234 29546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:13.716351 29546 leader_election.cc:302] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ff15424a9d54e6fbb43bde3ab200236; no voters:
I20260619 20:47:13.716550 29546 leader_election.cc:288] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:47:13.716753 29552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:47:13.716835 29546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:47:13.716958 29552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 LEADER]: Becoming Leader. State: Replica: 9ff15424a9d54e6fbb43bde3ab200236, State: Running, Role: LEADER
I20260619 20:47:13.717113 29552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:13.717624 29551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ff15424a9d54e6fbb43bde3ab200236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } } }
I20260619 20:47:13.717731 29551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: This master's current role is: LEADER
I20260619 20:47:13.717620 29552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ff15424a9d54e6fbb43bde3ab200236. Latest consensus state: current_term: 1 leader_uuid: "9ff15424a9d54e6fbb43bde3ab200236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } } }
I20260619 20:47:13.717976 29552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: This master's current role is: LEADER
I20260619 20:47:13.717959 29560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:47:13.718865 29560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:47:13.719640 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:13.708705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47952 (local address 127.13.195.126:45713)
0619 20:47:13.708909 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:13.708918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:13.708943 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:47:13.709363 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:13.709370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:13.709416 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:13.709567 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:13.710187 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:13.711171 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:13.712167 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:13.712775 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:13.712900 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:13.715517 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:13.715523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:13.715523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:13.715536 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:13.717760 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:13.718267 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:13.718270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:13.718271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:13.718372 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:13.718634 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:13.718635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:13.718636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:13.718974 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:13.719152 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:13.719273 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 20:47:13.719416 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260619 20:47:13.722836 29560 catalog_manager.cc:1347] Generated new cluster ID: 95e2f409d23448f2aa93a0dbb786aae2
I20260619 20:47:13.722887 29560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:47:13.730150 29560 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:47:13.730957 29560 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:47:13.736286 29560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Generated new TSK 0
I20260619 20:47:13.736496 29560 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:47:13.773635 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45713
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:47:13.855372 29572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:13.855574 29572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:13.855598 29572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:13.856791 29572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:13.856837 29572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:13.856894 29572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:47:13.858448 29572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:13.858705 29572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:13.858947 29572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:13.863955 29580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:13.863945 29578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:13.863947 29577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:13.864066 29572 server_base.cc:1034] running on GCE node
I20260619 20:47:13.864401 29572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:13.864662 29572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:13.865806 29572 hybrid_clock.cc:648] HybridClock initialized: now 1781902033865784 us; error 40 us; skew 500 ppm
Jun 19 20:47:13 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902033, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:13.868297 29572 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:47:13.868680 29572 webserver.cc:460] Webserver started at http://127.13.195.65:38761/ using document root <none> and password file <none>
I20260619 20:47:13.868925 29572 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:13.868970 29572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:13.869144 29572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:47:13.870281 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "2697e19123b741049833e5eecb44cd0a"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "6b3e438f3e2031c902d8c40b388869fc"
server_key_iv: "d424daecb1dadba200319bc0df1e62a0"
server_key_version: "encryptionkey@0"
I20260619 20:47:13.870667 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "2697e19123b741049833e5eecb44cd0a"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "6b3e438f3e2031c902d8c40b388869fc"
server_key_iv: "d424daecb1dadba200319bc0df1e62a0"
server_key_version: "encryptionkey@0"
I20260619 20:47:13.872040 29572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:47:13.872812 29587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:13.872982 29572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:47:13.873090 29572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "2697e19123b741049833e5eecb44cd0a"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "6b3e438f3e2031c902d8c40b388869fc"
server_key_iv: "d424daecb1dadba200319bc0df1e62a0"
server_key_version: "encryptionkey@0"
I20260619 20:47:13.873165 29572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:13.896831 29572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:13.899039 29572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:13.899230 29572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:13.899480 29572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:13.899806 29572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:47:13.899863 29572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:13.899906 29572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:47:13.899950 29572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:13.906044 29572 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:33657
I20260619 20:47:13.906095 29700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:33657 every 8 connection(s)
I20260619 20:47:13.906440 29572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:47:13.908735 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 29572
I20260619 20:47:13.908821 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:47:13.909036 14093 external_mini_cluster.cc:1428] Setting key 411469a5140a1be328f2ee2112a243d6
Jun 19 20:47:13 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902033, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:47:13.916944 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:13.907208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:43569 (local address 127.13.195.126:45713)
0619 20:47:13.907318 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:13.907321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:13.907506 (+ 185us) server_negotiation.cc:407] Connection header received
0619 20:47:13.908052 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:13.908053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:13.908065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:13.908118 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:13.909215 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:13.909876 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:13.911043 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:13.911288 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:13.911353 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:13.913835 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:13.913837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:13.913837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:13.913846 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:13.915359 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:13.915799 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:13.915800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:13.915801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:13.915863 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:13.916135 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:13.916136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:13.916136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:13.916308 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:13.916378 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:13.916695 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 20:47:13.916847 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 20:47:13.917114 29703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:13.907178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:43569)
0619 20:47:13.907369 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:13.907418 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:13.907938 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:13.908217 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:13.908221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:13.908855 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:13.909100 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:13.909107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:13.909978 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:13.909980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:13.910941 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:13.910945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:13.911378 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:13.911379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:13.911545 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:13.912101 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:13.912113 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:13.913725 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:13.915465 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:13.915473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:13.915475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:13.915712 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:13.915944 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:13.915945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:13.915945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:13.916059 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:13.916380 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:13.916388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:13.916555 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 20:47:13.916674 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:47:13.916856 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
I20260619 20:47:13.917829 29701 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45713
I20260619 20:47:13.917960 29701 heartbeater.cc:464] Registering TS with master...
I20260619 20:47:13.918169 29701 heartbeater.cc:511] Master 127.13.195.126:45713 requested a full tablet report, sending...
I20260619 20:47:13.918769 29510 ts_manager.cc:194] Registered new tserver with Master: 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657)
I20260619 20:47:13.919453 29510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:43569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:47:13.935520 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45713
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:47:14.017668 29706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:14.017880 29706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:14.017905 29706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:14.019289 29706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:14.019343 29706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:14.019372 29706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:47:14.020963 29706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:14.021247 29706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:14.021512 29706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:14.026698 29711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:14.026741 29714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:14.026875 29712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:14.026710 29706 server_base.cc:1034] running on GCE node
I20260619 20:47:14.027134 29706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:14.027390 29706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:14.028558 29706 hybrid_clock.cc:648] HybridClock initialized: now 1781902034028553 us; error 37 us; skew 500 ppm
Jun 19 20:47:14 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902034, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:14.031046 29706 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:47:14.031486 29706 webserver.cc:460] Webserver started at http://127.13.195.66:36597/ using document root <none> and password file <none>
I20260619 20:47:14.031668 29706 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:14.031708 29706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:14.031805 29706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:47:14.032811 29706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "d8643be7a94746e892b20b9e40b4aa13"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "efd8746010713d9909576ad9f7cdae0b"
server_key_iv: "f13eed1b04501ed1f6842ea5455f50b7"
server_key_version: "encryptionkey@0"
I20260619 20:47:14.033107 29706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "d8643be7a94746e892b20b9e40b4aa13"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "efd8746010713d9909576ad9f7cdae0b"
server_key_iv: "f13eed1b04501ed1f6842ea5455f50b7"
server_key_version: "encryptionkey@0"
I20260619 20:47:14.034337 29706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:47:14.035108 29721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:14.035287 29706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:47:14.035404 29706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "d8643be7a94746e892b20b9e40b4aa13"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "efd8746010713d9909576ad9f7cdae0b"
server_key_iv: "f13eed1b04501ed1f6842ea5455f50b7"
server_key_version: "encryptionkey@0"
I20260619 20:47:14.035482 29706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:14.059419 29706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:14.061918 29706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:14.062093 29706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:14.062356 29706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:14.062721 29706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:47:14.062790 29706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:14.062824 29706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:47:14.062852 29706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:14.069008 29706 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:42839
I20260619 20:47:14.069165 29834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:42839 every 8 connection(s)
I20260619 20:47:14.069417 29706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:47:14.070755 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 29706
I20260619 20:47:14.070870 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:47:14.071081 14093 external_mini_cluster.cc:1428] Setting key c5f25e4a3a5b17b3237d40f3dde78421
Jun 19 20:47:14 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902034, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:47:14.080118 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.070190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:59543 (local address 127.13.195.126:45713)
0619 20:47:14.070280 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.070283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.070485 (+ 202us) server_negotiation.cc:407] Connection header received
0619 20:47:14.071275 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.071277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.071291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.071354 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:14.072526 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.073079 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.074150 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.074345 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.074424 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.076824 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:14.076826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:14.076827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:14.076836 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:14.078718 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:14.079117 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:14.079119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:14.079119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:14.079180 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:14.079424 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:14.079425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:14.079425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:14.079566 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:14.079637 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.079919 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.080008 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:47:14.080228 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.070150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:59543)
0619 20:47:14.070330 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.070366 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:14.071127 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:14.071429 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:14.071432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:14.072145 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:14.072432 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.072440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.073197 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.073198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.074039 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.074041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.074449 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.074450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.074584 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.075171 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:14.075180 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:14.076727 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:14.078803 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:14.078810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:14.078812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:14.079033 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:14.079257 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:14.079257 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:14.079258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:14.079355 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:14.079631 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:14.079635 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:14.079789 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 20:47:14.079889 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 20:47:14.080015 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260619 20:47:14.080802 29835 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45713
I20260619 20:47:14.080902 29835 heartbeater.cc:464] Registering TS with master...
I20260619 20:47:14.081049 29835 heartbeater.cc:511] Master 127.13.195.126:45713 requested a full tablet report, sending...
I20260619 20:47:14.081408 29510 ts_manager.cc:194] Registered new tserver with Master: d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839)
I20260619 20:47:14.081785 29510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:59543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:47:14.096863 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45713
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:47:14.181798 29840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:14.182052 29840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:14.182093 29840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:14.183414 29840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:14.183486 29840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:14.183535 29840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:47:14.185037 29840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:14.185326 29840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:14.185567 29840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:14.190943 29845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:14.191077 29848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:14.191247 29846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:14.191475 29840 server_base.cc:1034] running on GCE node
I20260619 20:47:14.191689 29840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:14.191939 29840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:14.193120 29840 hybrid_clock.cc:648] HybridClock initialized: now 1781902034193106 us; error 42 us; skew 500 ppm
Jun 19 20:47:14 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902034, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:14.195686 29840 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:47:14.196132 29840 webserver.cc:460] Webserver started at http://127.13.195.67:45899/ using document root <none> and password file <none>
I20260619 20:47:14.196381 29840 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:14.196450 29840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:14.196604 29840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:47:14.197661 29840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "17f42b7774a54ddf825e9cd62323ed5a"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "cf23f760df6873a46431da72d9048506"
server_key_iv: "91d1dead7404d5332fd0a373e52ff8f3"
server_key_version: "encryptionkey@0"
I20260619 20:47:14.198027 29840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "17f42b7774a54ddf825e9cd62323ed5a"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "cf23f760df6873a46431da72d9048506"
server_key_iv: "91d1dead7404d5332fd0a373e52ff8f3"
server_key_version: "encryptionkey@0"
I20260619 20:47:14.199443 29840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:47:14.200179 29855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:14.200344 29840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:47:14.200444 29840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "17f42b7774a54ddf825e9cd62323ed5a"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "cf23f760df6873a46431da72d9048506"
server_key_iv: "91d1dead7404d5332fd0a373e52ff8f3"
server_key_version: "encryptionkey@0"
I20260619 20:47:14.200527 29840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:14.212793 29840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:14.214775 29840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:14.214941 29840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:14.215179 29840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:14.215490 29840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:47:14.215548 29840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:14.215590 29840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:47:14.215617 29840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:14.222366 29840 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:38017
I20260619 20:47:14.222404 29968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:38017 every 8 connection(s)
I20260619 20:47:14.222808 29840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:47:14 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902034, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:47:14.231663 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.223579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:46813 (local address 127.13.195.126:45713)
0619 20:47:14.223645 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.223648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.223883 (+ 235us) server_negotiation.cc:407] Connection header received
0619 20:47:14.224349 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.224350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.224360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.224417 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:14.225347 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.225844 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.226882 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.227070 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.227124 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.229130 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:14.229131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:14.229132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:14.229139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:14.230374 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:14.230789 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:14.230790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:14.230790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:14.230842 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:14.231119 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:14.231120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:14.231120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:14.231228 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:14.231277 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.231526 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.231594 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:47:14.231853 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 29840
I20260619 20:47:14.231949 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:47:14.231966 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.223569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:46813)
0619 20:47:14.223749 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.223794 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:14.224269 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:14.224479 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:14.224483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:14.225064 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:14.225269 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.225273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.225927 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.225928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.226809 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.226811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.227140 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.227141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.227240 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.227702 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:14.227711 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:14.229043 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:14.230491 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:14.230498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:14.230500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:14.230727 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:14.230933 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:14.230934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:14.230935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:14.231054 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:14.231287 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:14.231292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:14.231417 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 20:47:14.231504 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:47:14.231700 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260619 20:47:14.232156 14093 external_mini_cluster.cc:1428] Setting key e509dd4af542598e4e1bf058f32eaf2c
I20260619 20:47:14.232771 29969 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45713
I20260619 20:47:14.232884 29969 heartbeater.cc:464] Registering TS with master...
I20260619 20:47:14.233088 29969 heartbeater.cc:511] Master 127.13.195.126:45713 requested a full tablet report, sending...
I20260619 20:47:14.233451 29510 ts_manager.cc:194] Registered new tserver with Master: 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017)
I20260619 20:47:14.233843 29510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:46813
I20260619 20:47:14.244027 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:47:14.251684 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.245376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.13.195.126:45713)
0619 20:47:14.245452 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.245454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.245581 (+ 127us) server_negotiation.cc:407] Connection header received
0619 20:47:14.245660 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.245661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.245671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.245766 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:14.246355 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.246921 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.247665 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.247883 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.247959 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.249234 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:14.249236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:14.249237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:14.249245 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:14.250390 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:14.250819 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:14.250820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:14.250821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:14.250879 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:14.251091 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:14.251092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:14.251092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:14.251245 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:14.251350 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.251494 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.251577 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:47:14.253232 29510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:47:14.254935 29510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:47:14.261327 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.257184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:33657 (local address 127.0.0.1:49528)
0619 20:47:14.257563 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.257572 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:14.257614 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:14.257866 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:14.257868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:14.257870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:14.257926 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.257927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.259045 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.259046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.260138 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.260140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.261027 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.261028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.261102 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.261116 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:47:14.261191 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:47:14.261242 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260619 20:47:14.261390 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.257302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49528 (local address 127.13.195.65:33657)
0619 20:47:14.257715 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.257718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.257733 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:47:14.257767 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.257768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.257781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.257849 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:14.258012 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.258931 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.260244 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.260949 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.261023 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.261081 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.261228 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.261301 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":86,"threads_started":1}
I20260619 20:47:14.261828 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.257184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:42839 (local address 127.0.0.1:36710)
0619 20:47:14.257483 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.257523 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:14.257597 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:14.258212 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:14.258213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:14.258216 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:14.258266 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.258268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.259584 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.259585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.260649 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.260652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.261597 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.261598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.261671 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.261675 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:47:14.261726 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:47:14.261764 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1,"spinlock_wait_cycles":384,"thread_start_us":91,"threads_started":1}
I20260619 20:47:14.262099 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.257301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.13.195.66:42839)
0619 20:47:14.257970 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.257973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.257980 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:47:14.258053 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.258055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.258072 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.258152 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:14.258352 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.259471 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.260736 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.261448 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.261510 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.261569 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.261977 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.262020 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":89,"threads_started":1}
I20260619 20:47:14.262113 29635 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 (DEFAULT_TABLE table=test-table [id=559f377948e446ecb3780f1127ff8041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:47:14.262559 29635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2ec67eced5442fa2f787f48e9baa77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:47:14.262800 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.257377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38017 (local address 127.0.0.1:50910)
0619 20:47:14.258257 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.258267 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:14.258313 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:14.258625 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:14.258627 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:14.258629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:14.258680 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.258681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.259819 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.259820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.260877 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.260880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.262566 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.262568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.262630 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.262634 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:47:14.262691 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:47:14.262731 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260619 20:47:14.262861 29769 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 (DEFAULT_TABLE table=test-table [id=559f377948e446ecb3780f1127ff8041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:47:14.263109 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.257479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.13.195.67:38017)
0619 20:47:14.257643 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.257649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.258381 (+ 732us) server_negotiation.cc:407] Connection header received
0619 20:47:14.258451 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.258454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.258470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.258541 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:14.258756 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.259727 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.260971 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.262070 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.262170 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.262232 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.262973 (+ 741us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.263025 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260619 20:47:14.263216 29769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2ec67eced5442fa2f787f48e9baa77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:47:14.265347 29986 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Bootstrap starting.
I20260619 20:47:14.265844 29987 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Bootstrap starting.
I20260619 20:47:14.266045 29986 tablet_bootstrap.cc:654] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Neither blocks nor log segments found. Creating new log.
I20260619 20:47:14.266245 29903 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 (DEFAULT_TABLE table=test-table [id=559f377948e446ecb3780f1127ff8041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:47:14.266536 29987 tablet_bootstrap.cc:654] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Neither blocks nor log segments found. Creating new log.
I20260619 20:47:14.266672 29903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2ec67eced5442fa2f787f48e9baa77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:47:14.266710 29986 log.cc:824] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:14.267131 29987 log.cc:824] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:14.267534 29986 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: No bootstrap required, opened a new log
I20260619 20:47:14.267607 29986 ts_tablet_manager.cc:1397] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:47:14.267890 29987 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: No bootstrap required, opened a new log
I20260619 20:47:14.267970 29987 ts_tablet_manager.cc:1397] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:47:14.269533 29987 raft_consensus.cc:348] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.269711 29987 raft_consensus.cc:374] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:47:14.269757 29987 raft_consensus.cc:729] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8643be7a94746e892b20b9e40b4aa13, State: Initialized, Role: FOLLOWER
I20260619 20:47:14.269692 29986 raft_consensus.cc:348] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.269873 29986 raft_consensus.cc:374] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:47:14.269928 29986 raft_consensus.cc:729] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2697e19123b741049833e5eecb44cd0a, State: Initialized, Role: FOLLOWER
I20260619 20:47:14.269886 29987 consensus_queue.cc:260] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.270049 29986 consensus_queue.cc:260] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.270148 29990 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Bootstrap starting.
I20260619 20:47:14.270219 29987 ts_tablet_manager.cc:1428] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:47:14.270347 29835 heartbeater.cc:503] Master 127.13.195.126:45713 was elected leader, sending a full tablet report...
I20260619 20:47:14.270344 29986 ts_tablet_manager.cc:1428] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:47:14.270526 29701 heartbeater.cc:503] Master 127.13.195.126:45713 was elected leader, sending a full tablet report...
I20260619 20:47:14.270865 29990 tablet_bootstrap.cc:654] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Neither blocks nor log segments found. Creating new log.
I20260619 20:47:14.271461 29990 log.cc:824] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:14.272123 29990 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: No bootstrap required, opened a new log
I20260619 20:47:14.272209 29990 ts_tablet_manager.cc:1397] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:47:14.273597 29990 raft_consensus.cc:348] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.273769 29990 raft_consensus.cc:374] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:47:14.273818 29990 raft_consensus.cc:729] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f42b7774a54ddf825e9cd62323ed5a, State: Initialized, Role: FOLLOWER
I20260619 20:47:14.273959 29990 consensus_queue.cc:260] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.274221 29990 ts_tablet_manager.cc:1428] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:47:14.274302 29969 heartbeater.cc:503] Master 127.13.195.126:45713 was elected leader, sending a full tablet report...
W20260619 20:47:14.319978 29836 tablet.cc:2369] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:47:14.407140 29702 tablet.cc:2369] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:47:14.473397 29970 tablet.cc:2369] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:47:14.578310 29991 raft_consensus.cc:482] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:47:14.578496 29991 raft_consensus.cc:504] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.578871 29991 leader_election.cc:288] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657), 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017)
I20260619 20:47:14.582963 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.579182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:58439 (local address 127.13.195.65:33657)
0619 20:47:14.579267 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.579269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.579280 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:47:14.579338 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.579340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.579350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.579420 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:14.579636 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.580390 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.581949 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.582540 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.582630 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.582639 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.582822 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.582894 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 20:47:14.582964 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.579293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:33861 (local address 127.13.195.67:38017)
0619 20:47:14.579376 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.579379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.579386 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:47:14.579411 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.579412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.579422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.579477 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:14.579636 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.580389 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.581949 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.582540 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.582631 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.582639 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.582822 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.582894 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 20:47:14.582979 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.579047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:33657 (local address 127.13.195.66:58439)
0619 20:47:14.579110 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.579124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:14.579196 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:14.579450 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:14.579451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:14.579469 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:14.579550 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.579552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.580484 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.580485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.581842 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.581845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.582646 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.582647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.582738 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.582742 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:47:14.582801 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:47:14.582866 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":124}
I20260619 20:47:14.582986 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.579179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38017 (local address 127.13.195.66:33861)
0619 20:47:14.579304 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.579314 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:14.579351 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:14.579493 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:14.579494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:14.579495 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:14.579550 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.579552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.580484 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.580485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.581841 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:14.581845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.582647 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.582647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:14.582738 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.582742 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:47:14.582801 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:47:14.582866 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":16,"thread_start_us":60,"threads_started":1}
I20260619 20:47:14.583393 29923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ec67eced5442fa2f787f48e9baa77" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f42b7774a54ddf825e9cd62323ed5a" is_pre_election: true
I20260619 20:47:14.583395 29655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ec67eced5442fa2f787f48e9baa77" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2697e19123b741049833e5eecb44cd0a" is_pre_election: true
I20260619 20:47:14.583575 29923 raft_consensus.cc:2454] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 0.
I20260619 20:47:14.583590 29655 raft_consensus.cc:2454] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 0.
I20260619 20:47:14.583858 29722 leader_election.cc:302] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f42b7774a54ddf825e9cd62323ed5a, d8643be7a94746e892b20b9e40b4aa13; no voters:
I20260619 20:47:14.583982 29991 raft_consensus.cc:2781] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:47:14.584036 29991 raft_consensus.cc:482] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:47:14.584066 29991 raft_consensus.cc:3037] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:47:14.584764 29991 raft_consensus.cc:504] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.584897 29991 leader_election.cc:288] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 election: Requested vote from peers 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657), 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017)
I20260619 20:47:14.585055 29655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ec67eced5442fa2f787f48e9baa77" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2697e19123b741049833e5eecb44cd0a"
I20260619 20:47:14.585060 29923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a2ec67eced5442fa2f787f48e9baa77" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f42b7774a54ddf825e9cd62323ed5a"
I20260619 20:47:14.585137 29923 raft_consensus.cc:3037] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:47:14.585139 29655 raft_consensus.cc:3037] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:47:14.585989 29923 raft_consensus.cc:2454] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 1.
I20260619 20:47:14.585989 29655 raft_consensus.cc:2454] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 1.
I20260619 20:47:14.586143 29722 leader_election.cc:302] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f42b7774a54ddf825e9cd62323ed5a, d8643be7a94746e892b20b9e40b4aa13; no voters:
I20260619 20:47:14.586258 29991 raft_consensus.cc:2781] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:47:14.586427 29991 raft_consensus.cc:686] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 LEADER]: Becoming Leader. State: Replica: d8643be7a94746e892b20b9e40b4aa13, State: Running, Role: LEADER
I20260619 20:47:14.586549 29991 consensus_queue.cc:237] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:14.587386 29510 catalog_manager.cc:5495] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66). New cstate: current_term: 1 leader_uuid: "d8643be7a94746e892b20b9e40b4aa13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } health_report { overall_health: HEALTHY } } }
I20260619 20:47:14.618750 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.615912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.13.195.66:42839)
0619 20:47:14.615990 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:14.615991 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:14.615999 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:47:14.616034 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:14.616034 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:14.616053 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:14.616114 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:47:14.616245 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.616775 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.617706 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:14.617998 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:14.618086 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:14.618284 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:47:14.618398 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:47:14.618562 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:14.618580 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 20:47:14.618642 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:47:14.620596 29923 raft_consensus.cc:1261] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Refusing update from remote peer d8643be7a94746e892b20b9e40b4aa13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:47:14.620604 29655 raft_consensus.cc:1261] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Refusing update from remote peer d8643be7a94746e892b20b9e40b4aa13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:47:14.621001 29991 consensus_queue.cc:1035] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:47:14.621178 29996 consensus_queue.cc:1035] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:47:14.624504 29999 mvcc.cc:205] Tried to move back new op lower bound from 7298670733802897408 to 7298670733667151872. Current Snapshot: MvccSnapshot[applied={T|T < 7298670733802897408 or (T in {7298670733802897408})}]
I20260619 20:47:14.624625 30001 mvcc.cc:205] Tried to move back new op lower bound from 7298670733802897408 to 7298670733667151872. Current Snapshot: MvccSnapshot[applied={T|T < 7298670733802897408 or (T in {7298670733802897408})}]
I20260619 20:47:14.624574 30000 mvcc.cc:205] Tried to move back new op lower bound from 7298670733802897408 to 7298670733667151872. Current Snapshot: MvccSnapshot[applied={T|T < 7298670733802897408 or (T in {7298670733802897408})}]
I20260619 20:47:14.625855 29509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:47:14.625946 29509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:47:14.626696 29509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Sending DeleteTablet for 3 replicas of tablet 6a2ec67eced5442fa2f787f48e9baa77
I20260619 20:47:14.627018 29635 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:14 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:49528
I20260619 20:47:14.627024 29769 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:14 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:36710
I20260619 20:47:14.627045 29903 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:14 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:50910
I20260619 20:47:14.627255 30009 tablet_replica.cc:330] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: stopping tablet replica
I20260619 20:47:14.627306 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 29572
I20260619 20:47:14.627377 30009 raft_consensus.cc:2229] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:47:14.627552 30009 raft_consensus.cc:2258] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:47:14.634583 30010 tablet_replica.cc:330] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: stopping tablet replica
I20260619 20:47:14.634578 30011 tablet_replica.cc:330] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: stopping tablet replica
I20260619 20:47:14.634737 30011 raft_consensus.cc:2229] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:47:14.634747 30010 raft_consensus.cc:2229] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:47:14.634863 30010 raft_consensus.cc:2258] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:47:14.634920 30011 raft_consensus.cc:2258] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 20:47:14.635041 29494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:33657 (error 108)
I20260619 20:47:14.635485 30010 ts_tablet_manager.cc:1905] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:47:14.635491 30011 ts_tablet_manager.cc:1905] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:47:14.635630 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 29706
I20260619 20:47:14.637389 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.637243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:33657 (local address 127.0.0.1:49538)
0619 20:47:14.637309 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.637360 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 20:47:14.637581 29494 catalog_manager.cc:4584] TS 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a2ec67eced5442fa2f787f48e9baa77: Network error: Client connection negotiation failed: client connection to 127.13.195.65:33657: connect: Connection refused (error 111)
I20260619 20:47:14.640237 30010 ts_tablet_manager.cc:1918] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:47:14.640337 30010 log.cc:1196] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/6a2ec67eced5442fa2f787f48e9baa77
I20260619 20:47:14.640625 30010 ts_tablet_manager.cc:1939] T 6a2ec67eced5442fa2f787f48e9baa77 P 17f42b7774a54ddf825e9cd62323ed5a: Deleting consensus metadata
I20260619 20:47:14.640820 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:14.640706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:42839 (local address 127.0.0.1:36720)
0619 20:47:14.640761 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:14.640788 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.66:42839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:47:14.640985 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 29840
I20260619 20:47:14.641108 29494 catalog_manager.cc:4841] TS 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017): tablet 6a2ec67eced5442fa2f787f48e9baa77 (table test-table [id=559f377948e446ecb3780f1127ff8041]) successfully deleted
I20260619 20:47:14.645524 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 29478
I20260619 20:47:34.651535 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45713
--webserver_interface=127.13.195.126
--webserver_port=43699
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:47:34.731762 30013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:34.732012 30013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:34.732048 30013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:34.733181 30013 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:47:34.733251 30013 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:47:34.733276 30013 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:47:34.733305 30013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:47:34.733352 30013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:34.733384 30013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:34.733407 30013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:47:34.733433 30013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:47:34.734995 30013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=43699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:34.735273 30013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:34.735503 30013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:34.744210 30020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:34.744212 30019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:34.744215 30022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:34.744458 30013 server_base.cc:1034] running on GCE node
I20260619 20:47:34.744940 30013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:34.745329 30013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:34.746528 30013 hybrid_clock.cc:648] HybridClock initialized: now 1781902054746498 us; error 34 us; skew 500 ppm
Jun 19 20:47:34 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902054, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:34.749163 30013 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:47:34.749622 30013 webserver.cc:460] Webserver started at http://127.13.195.126:43699/ using document root <none> and password file <none>
I20260619 20:47:34.749863 30013 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:34.749949 30013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:34.751406 30013 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:34.752043 30029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:34.752218 30013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:47:34.752331 30013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "9ff15424a9d54e6fbb43bde3ab200236"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "a89d76965ad7fe1aa84a6067c44d9f24"
server_key_iv: "fcd0d8d1ab91faf417d50a10cf99c74f"
server_key_version: "encryptionkey@0"
I20260619 20:47:34.752744 30013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:34.769287 30013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:34.771374 30013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:34.771541 30013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:34.775736 30013 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:45713
I20260619 20:47:34.775780 30081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:45713 every 8 connection(s)
I20260619 20:47:34.776209 30013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:47:34.776552 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30013
I20260619 20:47:34.776836 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:33657
--local_ip_for_outbound_sockets=127.13.195.65
--tserver_master_addrs=127.13.195.126:45713
--webserver_port=38761
--webserver_interface=127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:47:34.778179 30082 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:47:34.779322 30082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Bootstrap starting.
I20260619 20:47:34.788501 30082 log.cc:824] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:34.790941 30082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:47:34.791301 30082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Bootstrap complete.
I20260619 20:47:34.793809 30082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:34.794078 30082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff15424a9d54e6fbb43bde3ab200236, State: Initialized, Role: FOLLOWER
I20260619 20:47:34.794278 30082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:34.794348 30082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:47:34.794417 30082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:47:34.794495 30082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:47:34.795608 30082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:34.795701 30082 leader_election.cc:302] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ff15424a9d54e6fbb43bde3ab200236; no voters:
I20260619 20:47:34.795943 30082 leader_election.cc:288] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:47:34.795984 30086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:47:34.796190 30086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 LEADER]: Becoming Leader. State: Replica: 9ff15424a9d54e6fbb43bde3ab200236, State: Running, Role: LEADER
I20260619 20:47:34.796233 30082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:47:34.796326 30086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:34.796623 30086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ff15424a9d54e6fbb43bde3ab200236. Latest consensus state: current_term: 2 leader_uuid: "9ff15424a9d54e6fbb43bde3ab200236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } } }
I20260619 20:47:34.796712 30086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: This master's current role is: LEADER
I20260619 20:47:34.796626 30087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9ff15424a9d54e6fbb43bde3ab200236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } } }
I20260619 20:47:34.797219 30091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:47:34.797223 30087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: This master's current role is: LEADER
I20260619 20:47:34.798158 30091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:47:34.798285 30091 catalog_manager.cc:1259] Loaded cluster ID: 95e2f409d23448f2aa93a0dbb786aae2
I20260619 20:47:34.798321 30091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:47:34.799300 30091 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:47:34.799505 30091 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Loaded TSK: 0
I20260619 20:47:34.816159 30091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Generated new TSK 1
I20260619 20:47:34.816308 30091 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:47:34.874548 30084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:34.874788 30084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:34.874830 30084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:34.875993 30084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:34.876063 30084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:34.876109 30084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:47:34.877550 30084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:33657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=38761
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:34.877823 30084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:34.878067 30084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:34.883193 30109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:34.883183 30112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:34.883179 30110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:34.883208 30084 server_base.cc:1034] running on GCE node
I20260619 20:47:34.883610 30084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:34.883867 30084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:34.885037 30084 hybrid_clock.cc:648] HybridClock initialized: now 1781902054885016 us; error 38 us; skew 500 ppm
Jun 19 20:47:34 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902054, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:34.887833 30084 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:47:34.888422 30084 webserver.cc:460] Webserver started at http://127.13.195.65:38761/ using document root <none> and password file <none>
I20260619 20:47:34.888705 30084 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:34.888798 30084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:34.890348 30084 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:47:34.891193 30119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:34.891404 30084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:34.891479 30084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "2697e19123b741049833e5eecb44cd0a"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "6b3e438f3e2031c902d8c40b388869fc"
server_key_iv: "d424daecb1dadba200319bc0df1e62a0"
server_key_version: "encryptionkey@0"
I20260619 20:47:34.891891 30084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:34.928421 30084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:34.931100 30084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:34.931270 30084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:34.931586 30084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:34.932106 30126 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:47:34.933938 30084 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:47:34.934118 30084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:47:34.934171 30084 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:47:34.934926 30084 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:47:34.934989 30084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:34.935035 30126 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Bootstrap starting.
I20260619 20:47:34.942919 30084 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:33657
I20260619 20:47:34.943017 30233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:33657 every 8 connection(s)
I20260619 20:47:34.943398 30084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:47:34.945688 30126 log.cc:824] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:34.948675 30126 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:47:34.949004 30126 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Bootstrap complete.
I20260619 20:47:34.950234 30126 ts_tablet_manager.cc:1397] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260619 20:47:34.951339 30126 raft_consensus.cc:348] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:34.951623 30126 raft_consensus.cc:729] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2697e19123b741049833e5eecb44cd0a, State: Initialized, Role: FOLLOWER
I20260619 20:47:34.951797 30126 consensus_queue.cc:260] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
Jun 19 20:47:34 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902054, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:47:34.952119 30126 ts_tablet_manager.cc:1428] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
I20260619 20:47:34.952907 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30084
I20260619 20:47:34.953223 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:42839
--local_ip_for_outbound_sockets=127.13.195.66
--tserver_master_addrs=127.13.195.126:45713
--webserver_port=36597
--webserver_interface=127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:47:34.956212 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:34.944343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:47773 (local address 127.13.195.126:45713)
0619 20:47:34.944613 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:34.944624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:34.944669 (+ 45us) server_negotiation.cc:407] Connection header received
0619 20:47:34.945356 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:34.945363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:34.945424 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:34.945600 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:34.946589 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.947969 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.949243 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.949957 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.950076 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:34.952644 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:34.952648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:34.952648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:34.952659 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:34.954409 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:34.954877 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:34.954880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:34.954880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:34.954974 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:34.955233 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:34.955235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:34.955235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:34.955499 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:34.955591 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:34.955904 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 20:47:34.956033 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260619 20:47:34.956212 30236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:34.944255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:47773)
0619 20:47:34.944468 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:34.944521 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:34.945110 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:34.945660 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:34.945664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:34.946117 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:34.946388 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:34.946394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.948121 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.948122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:34.949158 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:34.949160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.950069 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.950070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:34.950175 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:34.950855 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:34.950864 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:34.952488 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:34.954535 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:34.954541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:34.954544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:34.954774 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:34.955053 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:34.955054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:34.955054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:34.955166 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:34.955581 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:34.955588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:34.955741 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 20:47:34.955851 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 20:47:34.956022 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260619 20:47:34.957315 30234 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45713
I20260619 20:47:34.957408 30234 heartbeater.cc:464] Registering TS with master...
I20260619 20:47:34.957573 30234 heartbeater.cc:511] Master 127.13.195.126:45713 requested a full tablet report, sending...
I20260619 20:47:34.958290 30046 ts_manager.cc:194] Registered new tserver with Master: 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657)
I20260619 20:47:34.959513 30046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:47773
I20260619 20:47:34.960316 30234 heartbeater.cc:503] Master 127.13.195.126:45713 was elected leader, sending a full tablet report...
Jun 19 20:47:34 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902054, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.65@KRBTEST.COM
I20260619 20:47:34.968385 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:34.959005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56082 (local address 127.13.195.65:33657)
0619 20:47:34.959193 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:34.959197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:34.959345 (+ 148us) server_negotiation.cc:407] Connection header received
0619 20:47:34.959457 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:34.959459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:34.959473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:34.959548 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:34.960439 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.961756 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.962548 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.963120 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.963211 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:34.965505 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:34.965512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:34.965512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:34.965518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:34.967107 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:34.967582 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:34.967584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:34.967585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:34.967655 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:34.967859 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:34.967860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:34.967860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:34.967982 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:34.968041 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:34.968254 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:47:34.968325 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260619 20:47:34.968479 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:34.958914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:33657 (local address 127.0.0.1:56082)
0619 20:47:34.959221 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:34.959266 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:34.959360 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:34.959576 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:34.959582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:34.960255 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:34.960328 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:34.960332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.961845 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.961846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:34.962395 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:34.962397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:34.963206 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:34.963207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:34.963261 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:34.963765 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:34.963769 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:34.965421 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:34.967191 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:34.967199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:34.967201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:34.967491 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:34.967722 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:34.967723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:34.967723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:34.967795 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:34.968053 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:34.968060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:34.968117 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 20:47:34.968212 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 20:47:34.968348 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":83,"threads_started":1}
I20260619 20:47:34.969019 30150 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:14 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:56082
I20260619 20:47:34.969532 30243 tablet_replica.cc:330] stopping tablet replica
I20260619 20:47:34.969650 30243 raft_consensus.cc:2229] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:47:34.969717 30243 raft_consensus.cc:2258] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:47:34.969996 30243 ts_tablet_manager.cc:1905] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:47:34.972239 30243 ts_tablet_manager.cc:1918] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:47:34.972301 30243 log.cc:1196] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal/wals/6a2ec67eced5442fa2f787f48e9baa77
I20260619 20:47:34.972520 30243 ts_tablet_manager.cc:1939] T 6a2ec67eced5442fa2f787f48e9baa77 P 2697e19123b741049833e5eecb44cd0a: Deleting consensus metadata
I20260619 20:47:34.973093 30030 catalog_manager.cc:4841] TS 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657): tablet 6a2ec67eced5442fa2f787f48e9baa77 (table test-table [id=559f377948e446ecb3780f1127ff8041]) successfully deleted
W20260619 20:47:35.039927 30240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:35.040179 30240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:35.040215 30240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:35.041368 30240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:35.041442 30240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:35.041486 30240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:47:35.042980 30240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:42839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=36597
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:35.043267 30240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:35.043499 30240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:35.048666 30249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:35.048676 30248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:35.048738 30251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:35.048769 30240 server_base.cc:1034] running on GCE node
I20260619 20:47:35.049065 30240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:35.049304 30240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:35.050490 30240 hybrid_clock.cc:648] HybridClock initialized: now 1781902055050477 us; error 45 us; skew 500 ppm
Jun 19 20:47:35 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902055, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:35.053042 30240 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:47:35.053442 30240 webserver.cc:460] Webserver started at http://127.13.195.66:36597/ using document root <none> and password file <none>
I20260619 20:47:35.053668 30240 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:35.053725 30240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:35.055130 30240 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:35.055955 30258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:35.056159 30240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:35.056246 30240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "d8643be7a94746e892b20b9e40b4aa13"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "efd8746010713d9909576ad9f7cdae0b"
server_key_iv: "f13eed1b04501ed1f6842ea5455f50b7"
server_key_version: "encryptionkey@0"
I20260619 20:47:35.056588 30240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:35.074769 30240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:35.077188 30240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:35.077345 30240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:35.077613 30240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:35.078078 30265 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:47:35.079773 30240 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:47:35.079838 30240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:47:35.079881 30240 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:47:35.080574 30240 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:47:35.080634 30240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:47:35.080698 30265 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Bootstrap starting.
I20260619 20:47:35.082314 30265 log.cc:824] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:35.084721 30265 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:47:35.085017 30265 tablet_bootstrap.cc:492] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Bootstrap complete.
I20260619 20:47:35.085189 30265 ts_tablet_manager.cc:1397] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 20:47:35.086817 30265 raft_consensus.cc:348] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.087097 30265 raft_consensus.cc:729] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8643be7a94746e892b20b9e40b4aa13, State: Initialized, Role: FOLLOWER
I20260619 20:47:35.087311 30265 consensus_queue.cc:260] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.087677 30265 ts_tablet_manager.cc:1428] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
I20260619 20:47:35.087869 30240 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:42839
I20260619 20:47:35.087927 30374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:42839 every 8 connection(s)
I20260619 20:47:35.088232 30240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:47:35.088827 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30240
I20260619 20:47:35.089191 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:38017
--local_ip_for_outbound_sockets=127.13.195.67
--tserver_master_addrs=127.13.195.126:45713
--webserver_port=45899
--webserver_interface=127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:47:35 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902055, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:47:35.099490 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.088921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:32923 (local address 127.13.195.126:45713)
0619 20:47:35.088970 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.088973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.089264 (+ 291us) server_negotiation.cc:407] Connection header received
0619 20:47:35.089459 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.089460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.089481 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.089542 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:35.090522 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.091058 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.092761 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.092970 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.093025 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.096421 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:35.096423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:35.096423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:35.096432 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:35.098045 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.098467 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.098468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.098469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.098534 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.098819 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.098820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.098820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.098942 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:35.099017 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.099302 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.099395 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:47:35.099673 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.088911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:32923)
0619 20:47:35.089089 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:35.089138 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:35.089331 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:35.089638 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:35.089643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:35.090110 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:35.090410 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.090417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.091186 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.091187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.092666 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.092669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.093061 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.093062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.093238 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.093810 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:35.093830 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:35.096321 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:35.098129 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:35.098136 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:35.098139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:35.098372 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:35.098628 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:35.098629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:35.098629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:35.098745 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:35.099009 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:35.099016 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:35.099168 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 20:47:35.099269 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 20:47:35.099437 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260619 20:47:35.100193 30375 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45713
I20260619 20:47:35.100318 30375 heartbeater.cc:464] Registering TS with master...
I20260619 20:47:35.100539 30375 heartbeater.cc:511] Master 127.13.195.126:45713 requested a full tablet report, sending...
I20260619 20:47:35.101184 30046 ts_manager.cc:194] Registered new tserver with Master: d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839)
I20260619 20:47:35.101831 30046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:32923
Jun 19 20:47:35 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902054, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.66@KRBTEST.COM
I20260619 20:47:35.109686 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.101489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:42839 (local address 127.0.0.1:33176)
0619 20:47:35.101539 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:35.101550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:35.101594 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:35.102321 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:35.102322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:35.102509 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:35.102558 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.102560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.103643 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.103644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.104552 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.104554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.105003 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.105004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.105074 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.105444 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:35.105449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:35.106868 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:35.108626 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:35.108627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:35.108628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:35.108797 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:35.109012 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:35.109012 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:35.109013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:35.109083 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:35.109366 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:35.109367 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:35.109411 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:47:35.109486 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:47:35.109588 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:47:35.109728 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.101972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.13.195.66:42839)
0619 20:47:35.102142 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.102146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.102154 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:47:35.102206 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.102208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.102223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.102296 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:35.102640 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.103558 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.104640 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.104930 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.104981 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.106954 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:35.106963 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:35.106963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:35.106970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:35.108515 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.108879 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.108882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.108882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.108941 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.109148 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.109149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.109150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.109288 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:35.109375 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.109537 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.109627 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260619 20:47:35.110637 30308 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a2ec67eced5442fa2f787f48e9baa77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:14 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:33176
I20260619 20:47:35.111258 30380 tablet_replica.cc:330] stopping tablet replica
I20260619 20:47:35.111377 30380 raft_consensus.cc:2229] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:47:35.111446 30380 raft_consensus.cc:2258] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:47:35.111773 30380 ts_tablet_manager.cc:1905] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:47:35.114688 30380 ts_tablet_manager.cc:1918] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:47:35.114768 30380 log.cc:1196] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/6a2ec67eced5442fa2f787f48e9baa77
I20260619 20:47:35.115005 30380 ts_tablet_manager.cc:1939] T 6a2ec67eced5442fa2f787f48e9baa77 P d8643be7a94746e892b20b9e40b4aa13: Deleting consensus metadata
I20260619 20:47:35.115621 30032 catalog_manager.cc:4841] TS d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839): tablet 6a2ec67eced5442fa2f787f48e9baa77 (table test-table [id=559f377948e446ecb3780f1127ff8041]) successfully deleted
W20260619 20:47:35.185441 30378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:35.185657 30378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:35.185683 30378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:35.187003 30378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:35.187047 30378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:35.187076 30378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:47:35.188678 30378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:38017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=45899
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:35.188931 30378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:35.189184 30378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:35.194265 30385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:35.194267 30388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:35.194288 30386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:35.194733 30378 server_base.cc:1034] running on GCE node
I20260619 20:47:35.194981 30378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:35.195262 30378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:35.196434 30378 hybrid_clock.cc:648] HybridClock initialized: now 1781902055196424 us; error 32 us; skew 500 ppm
Jun 19 20:47:35 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902055, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:35.199501 30378 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:47:35.199945 30378 webserver.cc:460] Webserver started at http://127.13.195.67:45899/ using document root <none> and password file <none>
I20260619 20:47:35.200205 30378 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:35.200294 30378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:35.201617 30378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:35.202186 30395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:35.202345 30378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:47:35.202474 30378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "17f42b7774a54ddf825e9cd62323ed5a"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "cf23f760df6873a46431da72d9048506"
server_key_iv: "91d1dead7404d5332fd0a373e52ff8f3"
server_key_version: "encryptionkey@0"
I20260619 20:47:35.202818 30378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:35.215188 30378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:35.217509 30378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:35.217675 30378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:35.217902 30378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:35.218217 30378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:47:35.218271 30378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:35.218313 30378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:47:35.218340 30378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:35.223948 30378 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:38017
I20260619 20:47:35.224062 30508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:38017 every 8 connection(s)
I20260619 20:47:35.224370 30378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:47:35.225310 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30378
Jun 19 20:47:35 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902055, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:47:35.232868 30513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.225742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54088 (local address 127.13.195.126:45713)
0619 20:47:35.225849 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.225852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.225961 (+ 109us) server_negotiation.cc:407] Connection header received
0619 20:47:35.226027 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.226027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.226039 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.226106 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:35.226768 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.227551 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.228619 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.228925 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.228999 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.230230 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:35.230231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:35.230231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:35.230239 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:35.231689 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.232056 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.232057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.232057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.232142 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.232320 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.232321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.232322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.232452 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:35.232625 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.232666 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.232746 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":51,"threads_started":1}
I20260619 20:47:35.234902 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.225146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:52575 (local address 127.13.195.126:45713)
0619 20:47:35.225200 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.225201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.225506 (+ 305us) server_negotiation.cc:407] Connection header received
0619 20:47:35.226242 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.226243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.226252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.226285 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:35.227560 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.228330 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.229575 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.229778 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.229903 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.232161 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:35.232162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:35.232163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:35.232169 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:35.233750 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.234107 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.234108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.234108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.234153 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.234355 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.234355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.234356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.234492 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:35.234557 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.234778 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.234841 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:47:35.235119 30511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.225161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:52575)
0619 20:47:35.225347 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:35.225412 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:35.226157 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:35.226376 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:35.226380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:35.227193 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:35.227452 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.227457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.228433 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.228434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.229484 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.229486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.229914 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.229915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.230047 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.230535 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:35.230543 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:35.232079 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:35.233811 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:35.233828 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:35.233830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:35.234038 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:35.234209 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:35.234210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:35.234210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:35.234302 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:35.234552 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:35.234555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:35.234668 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 20:47:35.234750 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:47:35.234868 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260619 20:47:35.235597 30509 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45713
I20260619 20:47:35.235707 30509 heartbeater.cc:464] Registering TS with master...
I20260619 20:47:35.235884 30509 heartbeater.cc:511] Master 127.13.195.126:45713 requested a full tablet report, sending...
I20260619 20:47:35.236256 30046 ts_manager.cc:194] Registered new tserver with Master: 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017)
I20260619 20:47:35.236589 30046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:52575
I20260619 20:47:35.244539 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:47:35.247612 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.244941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54100 (local address 127.13.195.126:45713)
0619 20:47:35.244982 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.244986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.245136 (+ 150us) server_negotiation.cc:407] Connection header received
0619 20:47:35.245222 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.245223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.245266 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.245351 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:47:35.245500 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.245980 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.246882 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.247070 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.247147 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.247326 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:47:35.247376 (+ 50us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 20:47:35.247534 (+ 158us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54100: token expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:47:35.247694 30237 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54100: token expired
W20260619 20:47:35.247740 29975 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.195.126:45713)
I20260619 20:47:35.253939 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.248020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.13.195.126:45713)
0619 20:47:35.248057 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.248059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.248067 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:47:35.248093 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.248093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.248102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.248156 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:35.248713 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.249160 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.249846 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.250041 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.250131 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.251372 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:35.251374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:35.251374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:35.251382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:35.252732 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.253109 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.253110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.253110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.253167 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:35.253380 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:35.253380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:35.253381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:35.253496 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:35.253599 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.253742 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.253826 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:47:35.255355 30046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:47:35.257126 30046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:47:35.259327 30308 tablet_service.cc:1467] Processing CreateTablet for tablet d1fdd3fe629a41afba8c20786f4eae30 (DEFAULT_TABLE table=test-table [id=0747c8180ddf48788b8c8fdd0ab35977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:47:35.259496 30150 tablet_service.cc:1467] Processing CreateTablet for tablet d1fdd3fe629a41afba8c20786f4eae30 (DEFAULT_TABLE table=test-table [id=0747c8180ddf48788b8c8fdd0ab35977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:47:35.259562 30308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1fdd3fe629a41afba8c20786f4eae30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:47:35.259732 30150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1fdd3fe629a41afba8c20786f4eae30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:47:35.261097 30126 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Bootstrap starting.
I20260619 20:47:35.261111 30265 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Bootstrap starting.
I20260619 20:47:35.261437 30126 tablet_bootstrap.cc:654] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Neither blocks nor log segments found. Creating new log.
I20260619 20:47:35.261557 30265 tablet_bootstrap.cc:654] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Neither blocks nor log segments found. Creating new log.
I20260619 20:47:35.262048 30126 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: No bootstrap required, opened a new log
I20260619 20:47:35.262099 30126 ts_tablet_manager.cc:1397] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:35.262465 30126 raft_consensus.cc:348] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.262544 30126 raft_consensus.cc:374] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:47:35.262588 30126 raft_consensus.cc:729] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2697e19123b741049833e5eecb44cd0a, State: Initialized, Role: FOLLOWER
I20260619 20:47:35.262646 30126 consensus_queue.cc:260] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.262761 30126 ts_tablet_manager.cc:1428] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:35.262807 30265 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: No bootstrap required, opened a new log
I20260619 20:47:35.262857 30265 ts_tablet_manager.cc:1397] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:47:35.263015 30265 raft_consensus.cc:348] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.263094 30265 raft_consensus.cc:374] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:47:35.263151 30265 raft_consensus.cc:729] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8643be7a94746e892b20b9e40b4aa13, State: Initialized, Role: FOLLOWER
I20260619 20:47:35.263211 30265 consensus_queue.cc:260] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.263305 30265 ts_tablet_manager.cc:1428] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:35.263362 30375 heartbeater.cc:503] Master 127.13.195.126:45713 was elected leader, sending a full tablet report...
I20260619 20:47:35.264912 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.259100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38017 (local address 127.0.0.1:48332)
0619 20:47:35.259161 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:35.259173 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:35.259215 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:35.261382 (+ 2167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:35.261383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:35.261385 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:35.261433 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.261434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.262735 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.262737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.263700 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.263702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.264709 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.264710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.264766 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.264782 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 20:47:35.264829 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:47:35.264862 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 20:47:35.264930 30515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.259316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.13.195.67:38017)
0619 20:47:35.261181 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.261187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.261198 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:47:35.261244 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.261248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.261267 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.261363 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:35.261503 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.262628 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.263916 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.264644 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.264714 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.264759 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.264841 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.264869 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1828,"thread_start_us":133,"threads_started":1}
I20260619 20:47:35.265671 30443 tablet_service.cc:1467] Processing CreateTablet for tablet d1fdd3fe629a41afba8c20786f4eae30 (DEFAULT_TABLE table=test-table [id=0747c8180ddf48788b8c8fdd0ab35977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:47:35.265983 30443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1fdd3fe629a41afba8c20786f4eae30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:47:35.268947 30518 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Bootstrap starting.
I20260619 20:47:35.269541 30518 tablet_bootstrap.cc:654] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Neither blocks nor log segments found. Creating new log.
I20260619 20:47:35.270066 30518 log.cc:824] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:35.271001 30518 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: No bootstrap required, opened a new log
I20260619 20:47:35.271087 30518 ts_tablet_manager.cc:1397] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:47:35.272428 30518 raft_consensus.cc:348] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.272578 30518 raft_consensus.cc:374] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:47:35.272603 30518 raft_consensus.cc:729] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f42b7774a54ddf825e9cd62323ed5a, State: Initialized, Role: FOLLOWER
I20260619 20:47:35.272744 30518 consensus_queue.cc:260] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.273006 30518 ts_tablet_manager.cc:1428] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:47:35.273046 30509 heartbeater.cc:503] Master 127.13.195.126:45713 was elected leader, sending a full tablet report...
W20260619 20:47:35.338840 30376 tablet.cc:2369] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:47:35.351671 30369 raft_consensus.cc:482] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:47:35.351857 30369 raft_consensus.cc:504] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.352198 30369 leader_election.cc:288] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017), 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657)
I20260619 20:47:35.356175 30515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.352584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:46099 (local address 127.13.195.67:38017)
0619 20:47:35.352651 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.352653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.352685 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:47:35.352794 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.352795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.352806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.352869 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:35.353051 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.353839 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.355363 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.355762 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.355819 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.355827 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.356047 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.356105 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:47:35.356165 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.352536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38017 (local address 127.13.195.66:46099)
0619 20:47:35.352625 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:35.352639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:35.352713 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:35.352894 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:35.352895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:35.352897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:35.352977 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.352978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.353923 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.353923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.355247 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.355249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.355841 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.355842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.355939 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.355967 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 20:47:35.356029 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:47:35.356077 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":2}
I20260619 20:47:35.356257 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.352757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:33943 (local address 127.13.195.65:33657)
0619 20:47:35.352810 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.352813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.352953 (+ 140us) server_negotiation.cc:407] Connection header received
0619 20:47:35.352993 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.352994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.353036 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.353102 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:47:35.353249 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.354000 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.355318 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.355862 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.355941 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.356048 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.356125 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.356173 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:47:35.356531 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.352660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:33657 (local address 127.13.195.66:33943)
0619 20:47:35.352877 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:35.352888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:35.352943 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:35.353124 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:35.353124 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:35.353126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:47:35.353177 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.353178 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.354074 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.354075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.355237 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:35.355240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.355971 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.355972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:35.356063 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.356067 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:47:35.356120 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:47:35.356158 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":21,"thread_start_us":92,"threads_started":1}
I20260619 20:47:35.356617 30463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fdd3fe629a41afba8c20786f4eae30" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f42b7774a54ddf825e9cd62323ed5a" is_pre_election: true
I20260619 20:47:35.356806 30463 raft_consensus.cc:2454] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 0.
I20260619 20:47:35.356840 30185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fdd3fe629a41afba8c20786f4eae30" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2697e19123b741049833e5eecb44cd0a" is_pre_election: true
I20260619 20:47:35.357013 30185 raft_consensus.cc:2454] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 0.
I20260619 20:47:35.357087 30259 leader_election.cc:302] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f42b7774a54ddf825e9cd62323ed5a, d8643be7a94746e892b20b9e40b4aa13; no voters:
I20260619 20:47:35.357214 30369 raft_consensus.cc:2781] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:47:35.357286 30369 raft_consensus.cc:482] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:47:35.357304 30369 raft_consensus.cc:3037] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:47:35.358013 30369 raft_consensus.cc:504] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.358125 30369 leader_election.cc:288] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 election: Requested vote from peers 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017), 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657)
I20260619 20:47:35.358314 30463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fdd3fe629a41afba8c20786f4eae30" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f42b7774a54ddf825e9cd62323ed5a"
I20260619 20:47:35.358361 30185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fdd3fe629a41afba8c20786f4eae30" candidate_uuid: "d8643be7a94746e892b20b9e40b4aa13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2697e19123b741049833e5eecb44cd0a"
I20260619 20:47:35.358392 30463 raft_consensus.cc:3037] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:47:35.358456 30185 raft_consensus.cc:3037] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:47:35.359333 30463 raft_consensus.cc:2454] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 1.
I20260619 20:47:35.359403 30185 raft_consensus.cc:2454] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8643be7a94746e892b20b9e40b4aa13 in term 1.
I20260619 20:47:35.359547 30259 leader_election.cc:302] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2697e19123b741049833e5eecb44cd0a, d8643be7a94746e892b20b9e40b4aa13; no voters:
I20260619 20:47:35.359653 30369 raft_consensus.cc:2781] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:47:35.359839 30369 raft_consensus.cc:686] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 1 LEADER]: Becoming Leader. State: Replica: d8643be7a94746e892b20b9e40b4aa13, State: Running, Role: LEADER
I20260619 20:47:35.359953 30369 consensus_queue.cc:237] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:35.360769 30045 catalog_manager.cc:5495] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66). New cstate: current_term: 1 leader_uuid: "d8643be7a94746e892b20b9e40b4aa13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } health_report { overall_health: HEALTHY } } }
I20260619 20:47:35.378620 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.376301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.13.195.66:42839)
0619 20:47:35.376365 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:35.376367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:35.376378 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:47:35.376424 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:35.376425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:35.376442 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:35.376507 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:47:35.376643 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.377132 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.377841 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:35.378062 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:35.378121 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:35.378260 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:47:35.378358 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:47:35.378412 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:35.378539 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 20:47:35.378560 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:47:35.380296 30463 raft_consensus.cc:1261] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Refusing update from remote peer d8643be7a94746e892b20b9e40b4aa13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:47:35.380298 30185 raft_consensus.cc:1261] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Refusing update from remote peer d8643be7a94746e892b20b9e40b4aa13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:47:35.380654 30522 consensus_queue.cc:1035] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:47:35.380801 30369 consensus_queue.cc:1035] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:47:35.382928 30524 mvcc.cc:205] Tried to move back new op lower bound from 7298670818834653184 to 7298670818755112960. Current Snapshot: MvccSnapshot[applied={T|T < 7298670818834653184}]
I20260619 20:47:35.382961 30526 mvcc.cc:205] Tried to move back new op lower bound from 7298670818834653184 to 7298670818755112960. Current Snapshot: MvccSnapshot[applied={T|T < 7298670818834653184}]
I20260619 20:47:35.383320 30525 mvcc.cc:205] Tried to move back new op lower bound from 7298670818834653184 to 7298670818755112960. Current Snapshot: MvccSnapshot[applied={T|T < 7298670818834653184}]
I20260619 20:47:35.385871 30046 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:47:35.385964 30046 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:47:35.386662 30046 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Sending DeleteTablet for 3 replicas of tablet d1fdd3fe629a41afba8c20786f4eae30
I20260619 20:47:35.386997 30150 tablet_service.cc:1514] Processing DeleteTablet for tablet d1fdd3fe629a41afba8c20786f4eae30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:35 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:56082
I20260619 20:47:35.387002 30308 tablet_service.cc:1514] Processing DeleteTablet for tablet d1fdd3fe629a41afba8c20786f4eae30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:35 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:33176
I20260619 20:47:35.387040 30443 tablet_service.cc:1514] Processing DeleteTablet for tablet d1fdd3fe629a41afba8c20786f4eae30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:47:35 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:48332
I20260619 20:47:35.387092 30243 tablet_replica.cc:330] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: stopping tablet replica
I20260619 20:47:35.387100 30380 tablet_replica.cc:330] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: stopping tablet replica
I20260619 20:47:35.387141 30243 raft_consensus.cc:2229] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:47:35.387153 30380 raft_consensus.cc:2229] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:47:35.387215 30243 raft_consensus.cc:2258] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:47:35.387235 30380 raft_consensus.cc:2258] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:47:35.387266 30534 tablet_replica.cc:330] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: stopping tablet replica
I20260619 20:47:35.387292 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 30084
I20260619 20:47:35.387353 30534 raft_consensus.cc:2229] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:47:35.387434 30534 raft_consensus.cc:2258] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:47:35.388101 30534 ts_tablet_manager.cc:1905] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:47:35.388363 30380 ts_tablet_manager.cc:1905] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:47:35.389842 30534 ts_tablet_manager.cc:1918] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:47:35.389907 30534 log.cc:1196] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/d1fdd3fe629a41afba8c20786f4eae30
I20260619 20:47:35.390192 30534 ts_tablet_manager.cc:1939] T d1fdd3fe629a41afba8c20786f4eae30 P 17f42b7774a54ddf825e9cd62323ed5a: Deleting consensus metadata
I20260619 20:47:35.390218 30380 ts_tablet_manager.cc:1918] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:47:35.390269 30380 log.cc:1196] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/d1fdd3fe629a41afba8c20786f4eae30
I20260619 20:47:35.390604 30030 catalog_manager.cc:4841] TS 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017): tablet d1fdd3fe629a41afba8c20786f4eae30 (table test-table [id=0747c8180ddf48788b8c8fdd0ab35977]) successfully deleted
I20260619 20:47:35.390676 30380 ts_tablet_manager.cc:1939] T d1fdd3fe629a41afba8c20786f4eae30 P d8643be7a94746e892b20b9e40b4aa13: Deleting consensus metadata
I20260619 20:47:35.390995 30032 catalog_manager.cc:4841] TS d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839): tablet d1fdd3fe629a41afba8c20786f4eae30 (table test-table [id=0747c8180ddf48788b8c8fdd0ab35977]) successfully deleted
W20260619 20:47:35.393870 30030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:33657 (error 108)
I20260619 20:47:35.394316 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:35.394202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:33657 (local address 127.0.0.1:38582)
0619 20:47:35.394241 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:35.394284 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 20:47:35.394304 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 30240
W20260619 20:47:35.398895 30030 catalog_manager.cc:4584] TS 2697e19123b741049833e5eecb44cd0a (127.13.195.65:33657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1fdd3fe629a41afba8c20786f4eae30: Network error: Client connection negotiation failed: client connection to 127.13.195.65:33657: connect: Connection refused (error 111)
I20260619 20:47:35.400275 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 30378
I20260619 20:47:35.404554 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 30013
I20260619 20:47:55.410686 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45713
--webserver_interface=127.13.195.126
--webserver_port=43699
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:47:55.492339 30536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:55.492561 30536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:55.492587 30536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:55.493798 30536 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:47:55.493840 30536 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:47:55.493853 30536 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:47:55.493865 30536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:47:55.493877 30536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:55.493891 30536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:55.493947 30536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:47:55.493963 30536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:47:55.495546 30536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=43699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:55.495838 30536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:55.496102 30536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:55.501266 30544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:55.501343 30541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:55.501374 30536 server_base.cc:1034] running on GCE node
W20260619 20:47:55.501273 30542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:55.501734 30536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:55.502034 30536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:55.503181 30536 hybrid_clock.cc:648] HybridClock initialized: now 1781902075503158 us; error 39 us; skew 500 ppm
Jun 19 20:47:55 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902075, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:55.505692 30536 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:47:55.506181 30536 webserver.cc:460] Webserver started at http://127.13.195.126:43699/ using document root <none> and password file <none>
I20260619 20:47:55.506407 30536 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:55.506531 30536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:55.507890 30536 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:55.508602 30551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.508797 30536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:55.508896 30536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "9ff15424a9d54e6fbb43bde3ab200236"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "a89d76965ad7fe1aa84a6067c44d9f24"
server_key_iv: "fcd0d8d1ab91faf417d50a10cf99c74f"
server_key_version: "encryptionkey@0"
I20260619 20:47:55.509276 30536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:55.536036 30536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:55.538009 30536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:55.538192 30536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:55.542263 30536 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:45713
I20260619 20:47:55.542301 30603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:45713 every 8 connection(s)
I20260619 20:47:55.542732 30536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:47:55.544173 30604 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:47:55.544921 30604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Bootstrap starting.
I20260619 20:47:55.545955 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30536
I20260619 20:47:55.546223 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:33657
--local_ip_for_outbound_sockets=127.13.195.65
--tserver_master_addrs=127.13.195.126:45713
--webserver_port=38761
--webserver_interface=127.13.195.65
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:47:55.553583 30604 log.cc:824] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:55.555967 30604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:47:55.556298 30604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Bootstrap complete.
I20260619 20:47:55.558590 30604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:55.558852 30604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff15424a9d54e6fbb43bde3ab200236, State: Initialized, Role: FOLLOWER
I20260619 20:47:55.559038 30604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:55.559116 30604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:47:55.559190 30604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:47:55.559243 30604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:47:55.560230 30604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:55.560349 30604 leader_election.cc:302] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ff15424a9d54e6fbb43bde3ab200236; no voters:
I20260619 20:47:55.560544 30604 leader_election.cc:288] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 20:47:55.560585 30608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 20:47:55.560702 30608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [term 3 LEADER]: Becoming Leader. State: Replica: 9ff15424a9d54e6fbb43bde3ab200236, State: Running, Role: LEADER
I20260619 20:47:55.560830 30604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:47:55.560847 30608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } }
I20260619 20:47:55.561113 30608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ff15424a9d54e6fbb43bde3ab200236. Latest consensus state: current_term: 3 leader_uuid: "9ff15424a9d54e6fbb43bde3ab200236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } } }
I20260619 20:47:55.561370 30608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: This master's current role is: LEADER
I20260619 20:47:55.561291 30609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9ff15424a9d54e6fbb43bde3ab200236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff15424a9d54e6fbb43bde3ab200236" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45713 } } }
I20260619 20:47:55.561695 30609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236 [sys.catalog]: This master's current role is: LEADER
I20260619 20:47:55.561829 30617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:47:55.562722 30617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:47:55.562893 30617 catalog_manager.cc:1259] Loaded cluster ID: 95e2f409d23448f2aa93a0dbb786aae2
I20260619 20:47:55.563073 30617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:47:55.563871 30617 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:47:55.564064 30617 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Loaded TSK: 0
I20260619 20:47:55.564122 30617 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9ff15424a9d54e6fbb43bde3ab200236: Loaded TSK: 1
F20260619 20:47:55.564335 30617 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f813f8f29ad google::LogMessage::Fail() at ??:0
@ 0x7f813f8f588a google::LogMessage::SendToLog() at ??:0
@ 0x7f813f8f24de google::LogMessage::Flush() at ??:0
@ 0x7f813f8f3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f814205f104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f814205cd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f814784c1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f814784c48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f814784c594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f8147812c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f814781cad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f81417c00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f81417b5954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f81426196db start_thread at ??:0
@ 0x7f813e3f061f clone at ??:0
W20260619 20:47:55.648249 30606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:55.648468 30606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:55.648504 30606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:55.649649 30606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:55.649712 30606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:55.649758 30606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:47:55.651361 30606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:33657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=38761
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:55.651650 30606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:55.651882 30606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:55.656886 30646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:55.656963 30645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:55.656919 30648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:55.657052 30606 server_base.cc:1034] running on GCE node
I20260619 20:47:55.657485 30606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:55.657778 30606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:55.658938 30606 hybrid_clock.cc:648] HybridClock initialized: now 1781902075658917 us; error 34 us; skew 500 ppm
Jun 19 20:47:55 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902075, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:55.661633 30606 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:47:55.662194 30606 webserver.cc:460] Webserver started at http://127.13.195.65:38761/ using document root <none> and password file <none>
I20260619 20:47:55.662544 30606 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:55.662599 30606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:55.664127 30606 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:55.664906 30655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.665167 30606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:55.665270 30606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "2697e19123b741049833e5eecb44cd0a"
format_stamp: "Formatted at 2026-06-19 20:47:13 on dist-test-slave-n2td"
server_key: "6b3e438f3e2031c902d8c40b388869fc"
server_key_iv: "d424daecb1dadba200319bc0df1e62a0"
server_key_version: "encryptionkey@0"
I20260619 20:47:55.665635 30606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:55.678607 30606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:55.680869 30606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:55.681033 30606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:55.681296 30606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:55.681725 30662 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:47:55.683718 30606 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:47:55.683810 30606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:47:55.683864 30606 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:47:55.684574 30606 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:47:55.684639 30606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:47:55.684717 30662 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Bootstrap starting.
I20260619 20:47:55.693094 30606 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:33657
I20260619 20:47:55.693147 30769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:33657 every 8 connection(s)
I20260619 20:47:55.693637 30606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:47:55.696463 30662 log.cc:824] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:47:55.698942 30662 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:47:55.699270 30662 tablet_bootstrap.cc:492] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Bootstrap complete.
I20260619 20:47:55.700201 30662 ts_tablet_manager.cc:1397] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260619 20:47:55.701246 30662 raft_consensus.cc:348] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:55.701500 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30606
I20260619 20:47:55.701555 30662 raft_consensus.cc:729] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2697e19123b741049833e5eecb44cd0a, State: Initialized, Role: FOLLOWER
I20260619 20:47:55.701709 30662 consensus_queue.cc:260] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:55.701789 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:42839
--local_ip_for_outbound_sockets=127.13.195.66
--tserver_master_addrs=127.13.195.126:45713
--webserver_port=36597
--webserver_interface=127.13.195.66
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:47:55.702039 30662 ts_tablet_manager.cc:1428] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260619 20:47:55.774389 30772 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:47:55.694506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:53719)
0619 20:47:55.694744 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.694800 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:55.695619 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:55.774157 (+ 78538us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
W20260619 20:47:55.774679 30770 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:45713 (0 consecutive failures): Network error: Failed to ping master at 127.13.195.126:45713: Client connection negotiation failed: client connection to 127.13.195.126:45713: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:47:55.775041 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.774913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:36759)
0619 20:47:55.774970 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.775013 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:47:55.775404 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.775317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:60483)
0619 20:47:55.775365 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.775388 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 20:47:55.775539 30770 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:47:55.795473 30775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:55.795687 30775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:55.795711 30775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:55.796890 30775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:55.796936 30775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:55.796993 30775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:47:55.798545 30775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:42839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=36597
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:55.798789 30775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:55.799028 30775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:55.804062 30781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:55.804098 30783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:55.804088 30780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:55.804119 30775 server_base.cc:1034] running on GCE node
I20260619 20:47:55.804461 30775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:55.804741 30775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:55.805889 30775 hybrid_clock.cc:648] HybridClock initialized: now 1781902075805864 us; error 41 us; skew 500 ppm
Jun 19 20:47:55 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902075, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:55.808391 30775 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:47:55.808884 30775 webserver.cc:460] Webserver started at http://127.13.195.66:36597/ using document root <none> and password file <none>
I20260619 20:47:55.809118 30775 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:55.809190 30775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:55.810804 30775 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:55.811448 30790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.811614 30775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:47:55.811719 30775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "d8643be7a94746e892b20b9e40b4aa13"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "efd8746010713d9909576ad9f7cdae0b"
server_key_iv: "f13eed1b04501ed1f6842ea5455f50b7"
server_key_version: "encryptionkey@0"
I20260619 20:47:55.812085 30775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:55.823171 30775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:55.825073 30775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:55.825243 30775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:55.825484 30775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:55.825799 30775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:47:55.825852 30775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.825887 30775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:47:55.825914 30775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.832171 30775 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:42839
I20260619 20:47:55.832211 30903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:42839 every 8 connection(s)
I20260619 20:47:55.832553 30775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:47:55.833539 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.833175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:56531)
0619 20:47:55.833327 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.833398 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
W20260619 20:47:55.833958 30904 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:45713 (0 consecutive failures): Network error: Failed to ping master at 127.13.195.126:45713: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
I20260619 20:47:55.834280 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.834172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:50411)
0619 20:47:55.834225 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.834250 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 20:47:55.834574 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.834491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:54731)
0619 20:47:55.834516 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.834555 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 20:47:55.834682 30904 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:47:55.837812 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30775
I20260619 20:47:55.838119 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:38017
--local_ip_for_outbound_sockets=127.13.195.67
--tserver_master_addrs=127.13.195.126:45713
--webserver_port=45899
--webserver_interface=127.13.195.67
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:47:55.922204 30907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:47:55.922456 30907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:47:55.922483 30907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:47:55.923714 30907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:47:55.923765 30907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:47:55.923794 30907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:47:55.925199 30907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:38017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=45899
--tserver_master_addrs=127.13.195.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:47:55.925414 30907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:47:55.925607 30907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:47:55.930814 30913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:47:55.930850 30915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:55.930994 30907 server_base.cc:1034] running on GCE node
W20260619 20:47:55.930850 30912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:47:55.931334 30907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:47:55.931573 30907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:47:55.932744 30907 hybrid_clock.cc:648] HybridClock initialized: now 1781902075932709 us; error 49 us; skew 500 ppm
Jun 19 20:47:55 dist-test-slave-n2td krb5kdc[29465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902075, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:47:55.935629 30907 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:47:55.936117 30907 webserver.cc:460] Webserver started at http://127.13.195.67:45899/ using document root <none> and password file <none>
I20260619 20:47:55.936311 30907 fs_manager.cc:359] Metadata directory not provided
I20260619 20:47:55.936362 30907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:47:55.937759 30907 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:55.938416 30922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.938714 30907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:47:55.938800 30907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "17f42b7774a54ddf825e9cd62323ed5a"
format_stamp: "Formatted at 2026-06-19 20:47:14 on dist-test-slave-n2td"
server_key: "cf23f760df6873a46431da72d9048506"
server_key_iv: "91d1dead7404d5332fd0a373e52ff8f3"
server_key_version: "encryptionkey@0"
I20260619 20:47:55.939185 30907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:47:55.962116 30907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:47:55.964568 30907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:47:55.964740 30907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:47:55.964994 30907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:47:55.965327 30907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:47:55.965387 30907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.965430 30907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:47:55.965458 30907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:47:55.971570 30907 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:38017
I20260619 20:47:55.971652 31035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:38017 every 8 connection(s)
I20260619 20:47:55.972007 30907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:47:55.973186 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.972719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:36139)
0619 20:47:55.972914 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.972997 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260619 20:47:55.973654 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 30907
W20260619 20:47:55.973675 31036 heartbeater.cc:644] Failed to heartbeat to 127.13.195.126:45713 (0 consecutive failures): Network error: Failed to ping master at 127.13.195.126:45713: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
I20260619 20:47:55.974012 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.973888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:42505)
0619 20:47:55.973947 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.973980 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:47:55.974392 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:55.974264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:34275)
0619 20:47:55.974316 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:55.974365 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 20:47:55.974556 31036 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:47:56.776166 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:56.775931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:57567)
0619 20:47:56.776064 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:56.776104 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260619 20:47:56.835212 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:56.834993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:53351)
0619 20:47:56.835120 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:56.835163 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260619 20:47:56.975077 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:56.974871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:45111)
0619 20:47:56.974989 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:56.975025 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260619 20:47:57.391418 31043 raft_consensus.cc:482] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:47:57.391533 31043 raft_consensus.cc:504] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:57.391937 31043 leader_election.cc:288] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017), d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839)
Jun 19 20:47:57 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902075, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.66@KRBTEST.COM
Jun 19 20:47:57 dist-test-slave-n2td krb5kdc[29465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902075, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.67@KRBTEST.COM
I20260619 20:47:57.403741 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:57.392153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:42839 (local address 127.13.195.65:48877)
0619 20:47:57.392420 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:57.392429 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:57.392469 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:57.393341 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:57.393344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:57.393870 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:57.394143 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:57.394151 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.395824 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.395825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:57.397210 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:57.397213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.397849 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.397851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:57.397970 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:57.398618 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:57.398626 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:57.400074 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:57.402049 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:57.402056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:57.402058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:57.402277 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:57.402616 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:57.402617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:57.402618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:57.402715 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:57.403205 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:57.403211 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:57.403344 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 20:47:57.403452 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 20:47:57.403630 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":86,"thread_start_us":48,"threads_started":1}
I20260619 20:47:57.403800 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:57.392129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:38017 (local address 127.13.195.65:45107)
0619 20:47:57.392274 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:57.392287 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:47:57.392346 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:47:57.393341 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:47:57.393344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:47:57.393870 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:47:57.394143 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:57.394151 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.395824 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.395825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:57.397210 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:47:57.397213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.397858 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.397859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:47:57.397971 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:57.398691 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:47:57.398695 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:47:57.400469 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:47:57.402521 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:57.402522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:57.402522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:57.402721 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:57.402955 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:47:57.402956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:47:57.402956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:47:57.403016 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:47:57.403448 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:47:57.403449 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:47:57.403491 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:47:57.403553 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:47:57.403683 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":167,"thread_start_us":69,"threads_started":1}
I20260619 20:47:57.403890 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:57.392259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:45107 (local address 127.13.195.67:38017)
0619 20:47:57.392424 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:57.392436 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:57.392474 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:47:57.393040 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:57.393043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:57.393103 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:57.393264 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:57.394278 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.395701 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.397334 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.397754 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.397876 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:57.400582 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:57.400585 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:57.400586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:57.400598 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:57.402397 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:57.402798 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:57.402801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:57.402801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:57.402886 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:57.403079 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:57.403080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:57.403080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:57.403351 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:57.403458 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:57.403632 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 20:47:57.403781 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":54,"threads_started":1}
I20260619 20:47:57.404173 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:57.392276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:48877 (local address 127.13.195.66:42839)
0619 20:47:57.392425 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 20:47:57.392436 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:47:57.392465 (+ 29us) server_negotiation.cc:407] Connection header received
0619 20:47:57.393040 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:47:57.393042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:47:57.393103 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:47:57.393264 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:47:57.394278 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.395701 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.397335 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:47:57.397751 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:47:57.397877 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:47:57.400193 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:47:57.400198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:47:57.400198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:47:57.400210 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:47:57.401949 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:57.402384 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:57.402387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:57.402388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:57.402542 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:47:57.402790 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:47:57.402791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:47:57.402791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:47:57.403116 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:47:57.403200 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 20:47:57.403916 (+ 716us) server_negotiation.cc:299] Negotiation successful
0619 20:47:57.404051 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
W20260619 20:47:57.404690 30656 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
W20260619 20:47:57.405064 30658 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
I20260619 20:47:57.405122 30658 leader_election.cc:302] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2697e19123b741049833e5eecb44cd0a; no voters: 17f42b7774a54ddf825e9cd62323ed5a, d8643be7a94746e892b20b9e40b4aa13
I20260619 20:47:57.405258 31043 raft_consensus.cc:2726] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:47:57.776770 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:57.776623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:46961)
0619 20:47:57.776698 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:57.776730 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 20:47:57.835878 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:57.835661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:35927)
0619 20:47:57.835782 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:57.835832 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260619 20:47:57.975773 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:57.975569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:42779)
0619 20:47:57.975696 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:57.975733 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260619 20:47:58.777420 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:58.777211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:46743)
0619 20:47:58.777342 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:58.777373 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260619 20:47:58.836603 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:58.836350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:59203)
0619 20:47:58.836483 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:58.836544 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260619 20:47:58.976413 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:58.976191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:44855)
0619 20:47:58.976331 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:58.976370 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260619 20:47:59.477347 31053 raft_consensus.cc:482] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:47:59.477438 31053 raft_consensus.cc:504] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:47:59.477597 31053 leader_election.cc:288] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017), d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839)
W20260619 20:47:59.478045 30658 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
W20260619 20:47:59.478106 30656 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
I20260619 20:47:59.478132 30658 leader_election.cc:302] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2697e19123b741049833e5eecb44cd0a; no voters: 17f42b7774a54ddf825e9cd62323ed5a, d8643be7a94746e892b20b9e40b4aa13
I20260619 20:47:59.478206 31053 raft_consensus.cc:2726] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:47:59.778139 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:59.777920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:44629)
0619 20:47:59.778047 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:59.778078 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260619 20:47:59.837416 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:59.837198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:33005)
0619 20:47:59.837317 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:59.837360 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260619 20:47:59.977065 31056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:47:59.976861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:43309)
0619 20:47:59.976983 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 20:47:59.977011 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260619 20:48:00.778973 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:00.778722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:40183)
0619 20:48:00.778882 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:00.778926 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260619 20:48:00.838114 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:00.837900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:36423)
0619 20:48:00.838019 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:00.838069 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260619 20:48:00.977742 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:00.977502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:42269)
0619 20:48:00.977655 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:00.977698 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260619 20:48:01.779736 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:01.779449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:34323)
0619 20:48:01.779599 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:01.779676 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260619 20:48:01.838898 31061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:01.838683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:43621)
0619 20:48:01.838820 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:01.838859 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260619 20:48:01.892874 31062 raft_consensus.cc:482] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:01.892978 31062 raft_consensus.cc:504] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:48:01.893196 31062 leader_election.cc:288] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017), d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839)
W20260619 20:48:01.893576 30658 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
W20260619 20:48:01.893662 30656 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
I20260619 20:48:01.893734 30656 leader_election.cc:302] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2697e19123b741049833e5eecb44cd0a; no voters: 17f42b7774a54ddf825e9cd62323ed5a, d8643be7a94746e892b20b9e40b4aa13
I20260619 20:48:01.893816 31062 raft_consensus.cc:2726] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:48:01.978662 31063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:01.978297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:39069)
0619 20:48:01.978418 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:01.978607 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260619 20:48:02.780432 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:02.780220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:48219)
0619 20:48:02.780349 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:02.780385 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260619 20:48:02.839685 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:02.839376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:35525)
0619 20:48:02.839572 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:02.839630 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":57,"threads_started":1}
I20260619 20:48:02.979322 31066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:02.979074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:56111)
0619 20:48:02.979240 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:02.979276 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260619 20:48:03.781239 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:03.780992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:45493)
0619 20:48:03.781150 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:03.781192 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":65,"threads_started":1}
I20260619 20:48:03.840265 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:03.840078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:35811)
0619 20:48:03.840190 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:03.840219 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260619 20:48:03.955042 31069 raft_consensus.cc:482] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:03.955159 31069 raft_consensus.cc:504] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17f42b7774a54ddf825e9cd62323ed5a" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 38017 } } peers { permanent_uuid: "2697e19123b741049833e5eecb44cd0a" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 33657 } } peers { permanent_uuid: "d8643be7a94746e892b20b9e40b4aa13" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42839 } }
I20260619 20:48:03.955312 31069 leader_election.cc:288] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017), d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839)
W20260619 20:48:03.955623 30656 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 17f42b7774a54ddf825e9cd62323ed5a (127.13.195.67:38017): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
W20260619 20:48:03.955696 30658 leader_election.cc:341] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d8643be7a94746e892b20b9e40b4aa13 (127.13.195.66:42839): Not found: Tablet not found: d1fdd3fe629a41afba8c20786f4eae30
I20260619 20:48:03.955773 30658 leader_election.cc:302] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2697e19123b741049833e5eecb44cd0a; no voters: 17f42b7774a54ddf825e9cd62323ed5a, d8643be7a94746e892b20b9e40b4aa13
I20260619 20:48:03.955859 31069 raft_consensus.cc:2726] T d1fdd3fe629a41afba8c20786f4eae30 P 2697e19123b741049833e5eecb44cd0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:48:03.980058 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:03.979824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:59035)
0619 20:48:03.979974 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:03.980013 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260619 20:48:04.781980 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:04.781751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:38529)
0619 20:48:04.781890 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:04.781935 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260619 20:48:04.840925 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:04.840725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:49249)
0619 20:48:04.840840 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:04.840880 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260619 20:48:04.980826 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:04.980595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:34857)
0619 20:48:04.980739 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:04.980781 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260619 20:48:05.782779 31074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:05.782556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:59517)
0619 20:48:05.782686 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:05.782728 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
I20260619 20:48:05.841701 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:05.841415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:51075)
0619 20:48:05.841614 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:05.841655 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":127,"threads_started":1}
I20260619 20:48:05.981570 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:05.981312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:53425)
0619 20:48:05.981454 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:05.981491 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260619 20:48:06.783447 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:06.783239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:50479)
0619 20:48:06.783361 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:06.783400 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260619 20:48:06.842306 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:06.842091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:44067)
0619 20:48:06.842225 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:06.842259 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260619 20:48:06.982285 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:06.982080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:36511)
0619 20:48:06.982205 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:06.982241 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260619 20:48:07.784134 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:07.783913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:56865)
0619 20:48:07.784042 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:07.784084 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260619 20:48:07.842924 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:07.842724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:40631)
0619 20:48:07.842839 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:07.842878 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260619 20:48:07.982951 31082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:07.982717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:55333)
0619 20:48:07.982864 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:07.982905 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260619 20:48:08.784884 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:08.784640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:57623)
0619 20:48:08.784794 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:08.784837 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260619 20:48:08.843652 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:08.843391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:49535)
0619 20:48:08.843542 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:08.843604 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260619 20:48:08.983642 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:08.983412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:32931)
0619 20:48:08.983555 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:08.983595 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260619 20:48:09.785655 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:09.785384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:53985)
0619 20:48:09.785542 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:09.785587 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260619 20:48:09.844336 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:09.844123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:52937)
0619 20:48:09.844252 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:09.844291 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260619 20:48:09.984416 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:09.984204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:33225)
0619 20:48:09.984329 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:09.984369 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260619 20:48:10.786705 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:10.786423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.65:45539)
0619 20:48:10.786618 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:10.786662 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260619 20:48:10.845103 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:10.844876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.66:40037)
0619 20:48:10.845018 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:10.845055 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260619 20:48:10.977870 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 30606
I20260619 20:48:10.982821 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 30775
I20260619 20:48:10.985177 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:10.984966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45713 (local address 127.13.195.67:53561)
0619 20:48:10.985092 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:10.985135 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260619 20:48:10.987421 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 30907
2026-06-19T20:48:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58603 ms)
[----------] 30 tests from SecurityITest (233747 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:11 dist-test-slave-n2td krb5kdc[31094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:11 dist-test-slave-n2td krb5kdc[31094](info): set up 2 sockets
Jun 19 20:48:11 dist-test-slave-n2td krb5kdc[31094](info): commencing operation
krb5kdc: starting...
W20260619 20:48:12.036985 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:12.152992 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40631
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:40631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:12.232595 31107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:12.232841 31107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:12.232877 31107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:12.233986 31107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:12.234058 31107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:12.234082 31107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:12.234123 31107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:12.234148 31107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:12.235613 31107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:40631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:12.235886 31107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:12.236122 31107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:12.241240 31113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:12.241289 31115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:12.241245 31112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:12.241297 31107 server_base.cc:1034] running on GCE node
I20260619 20:48:12.241640 31107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:12.241923 31107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:12.243060 31107 hybrid_clock.cc:648] HybridClock initialized: now 1781902092243043 us; error 28 us; skew 500 ppm
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:12.245486 31107 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:12.245898 31107 webserver.cc:460] Webserver started at http://127.13.195.126:38909/ using document root <none> and password file <none>
I20260619 20:48:12.246093 31107 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:12.246141 31107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:12.246229 31107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:12.247303 31107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "e71125030f3d4584b4a846e1e0f3a622"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "ba1f21c4338a5d93b51addbc95d7bfa1"
server_key_iv: "edb30008c801263418905792452533c0"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.247591 31107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "e71125030f3d4584b4a846e1e0f3a622"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "ba1f21c4338a5d93b51addbc95d7bfa1"
server_key_iv: "edb30008c801263418905792452533c0"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.248852 31107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:48:12.249518 31122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.249692 31107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:12.249794 31107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "e71125030f3d4584b4a846e1e0f3a622"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "ba1f21c4338a5d93b51addbc95d7bfa1"
server_key_iv: "edb30008c801263418905792452533c0"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.249863 31107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:12.261278 31107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:12.263329 31107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:12.263484 31107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:12.267477 31107 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:40631
I20260619 20:48:12.267508 31174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:40631 every 8 connection(s)
I20260619 20:48:12.267845 31107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:12.268728 31175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:12.270973 31175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622: Bootstrap starting.
I20260619 20:48:12.271559 31175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:12.271855 31175 log.cc:824] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:12.272565 31175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622: No bootstrap required, opened a new log
I20260619 20:48:12.274199 31175 raft_consensus.cc:348] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71125030f3d4584b4a846e1e0f3a622" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40631 } }
I20260619 20:48:12.274334 31175 raft_consensus.cc:374] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:12.274381 31175 raft_consensus.cc:729] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e71125030f3d4584b4a846e1e0f3a622, State: Initialized, Role: FOLLOWER
I20260619 20:48:12.274551 31175 consensus_queue.cc:260] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71125030f3d4584b4a846e1e0f3a622" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40631 } }
I20260619 20:48:12.274621 31175 raft_consensus.cc:388] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:12.274664 31175 raft_consensus.cc:482] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:12.274713 31175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:12.275245 31175 raft_consensus.cc:504] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71125030f3d4584b4a846e1e0f3a622" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40631 } }
I20260619 20:48:12.275358 31175 leader_election.cc:302] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e71125030f3d4584b4a846e1e0f3a622; no voters:
I20260619 20:48:12.275553 31175 leader_election.cc:288] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:12.275605 31178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:12.275784 31178 raft_consensus.cc:686] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [term 1 LEADER]: Becoming Leader. State: Replica: e71125030f3d4584b4a846e1e0f3a622, State: Running, Role: LEADER
I20260619 20:48:12.275815 31175 sys_catalog.cc:564] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:12.275913 31178 consensus_queue.cc:237] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71125030f3d4584b4a846e1e0f3a622" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40631 } }
I20260619 20:48:12.276302 31180 sys_catalog.cc:455] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e71125030f3d4584b4a846e1e0f3a622. Latest consensus state: current_term: 1 leader_uuid: "e71125030f3d4584b4a846e1e0f3a622" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71125030f3d4584b4a846e1e0f3a622" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40631 } } }
I20260619 20:48:12.276384 31180 sys_catalog.cc:458] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:12.276437 31179 sys_catalog.cc:455] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e71125030f3d4584b4a846e1e0f3a622" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71125030f3d4584b4a846e1e0f3a622" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40631 } } }
I20260619 20:48:12.276495 31179 sys_catalog.cc:458] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:12.276610 31186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:12.277141 31186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:12.277308 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31107
I20260619 20:48:12.277438 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:12.277650 14093 external_mini_cluster.cc:1428] Setting key 90350bee19a077b99f30f796bffd958b
I20260619 20:48:12.282708 31186 catalog_manager.cc:1347] Generated new cluster ID: 20717bf6b10342618a22e9e0ff1e4e2a
I20260619 20:48:12.282785 31186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:12.288642 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.278063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59334 (local address 127.13.195.126:40631)
0619 20:48:12.278561 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.278570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.278603 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:48:12.278715 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.278720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.278762 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.278870 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:12.279613 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.280481 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.281605 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.282032 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.282119 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.284919 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:12.284925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:12.284925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:12.284941 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:12.286984 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.287377 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.287380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.287381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.287501 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.287698 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.287699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.287699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.288020 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:12.288206 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.288278 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.288416 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":66,"threads_started":1}
I20260619 20:48:12.291653 31186 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:12.292326 31186 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:12.310648 31186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622: Generated new TSK 0
I20260619 20:48:12.310875 31186 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:48:12.346345 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40631
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:12.450544 31201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:12.450743 31201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:12.450766 31201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:12.451894 31201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:12.451937 31201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:12.451996 31201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:48:12.453472 31201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:12.453753 31201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:12.454000 31201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:12.459245 31209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:12.459213 31206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:12.459237 31207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:12.459545 31201 server_base.cc:1034] running on GCE node
I20260619 20:48:12.459784 31201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:12.460054 31201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:12.461206 31201 hybrid_clock.cc:648] HybridClock initialized: now 1781902092461194 us; error 30 us; skew 500 ppm
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:12.463897 31201 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:48:12.464322 31201 webserver.cc:460] Webserver started at http://127.13.195.65:45373/ using document root <none> and password file <none>
I20260619 20:48:12.464569 31201 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:12.464629 31201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:12.464807 31201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:12.466004 31201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "93644b508d5a44f199afb295d921dca6"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "8ad24e3a396f3aef151c20adb53a0d0d"
server_key_iv: "7c0b55171429a83c5188a30dee15383b"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.466363 31201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "93644b508d5a44f199afb295d921dca6"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "8ad24e3a396f3aef151c20adb53a0d0d"
server_key_iv: "7c0b55171429a83c5188a30dee15383b"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.467682 31201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:12.468410 31216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.468593 31201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:12.468694 31201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "93644b508d5a44f199afb295d921dca6"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "8ad24e3a396f3aef151c20adb53a0d0d"
server_key_iv: "7c0b55171429a83c5188a30dee15383b"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.468768 31201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:12.490751 31201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:12.492774 31201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:12.492950 31201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:12.493197 31201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:12.493526 31201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:12.493561 31201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.493583 31201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:12.493638 31201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.499542 31201 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:39323
I20260619 20:48:12.499594 31329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:39323 every 8 connection(s)
I20260619 20:48:12.499918 31201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:48:12.502004 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31201
I20260619 20:48:12.502097 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:48:12.502316 14093 external_mini_cluster.cc:1428] Setting key a0f86410134510c53f360a879f102727
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:12.510202 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.500644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:45303 (local address 127.13.195.126:40631)
0619 20:48:12.500705 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.500707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.500923 (+ 216us) server_negotiation.cc:407] Connection header received
0619 20:48:12.501469 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.501470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.501487 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.501538 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:12.502740 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.503476 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.504654 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.504852 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.504909 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.507281 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:12.507282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:12.507283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:12.507291 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:12.508708 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.509146 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.509148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.509148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.509206 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.509523 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.509524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.509524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.509657 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:12.509732 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.510017 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.510108 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:48:12.510411 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.500630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40631 (local address 127.13.195.65:45303)
0619 20:48:12.500801 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.500838 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.501357 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.501613 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.501616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.502303 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:12.502637 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.502644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.503574 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.503575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.504563 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.504566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.504939 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.504940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.505099 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.505722 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:12.505731 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:12.507193 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:12.508809 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:12.508817 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:12.508819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:12.509046 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:12.509322 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:12.509323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:12.509323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:12.509438 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:12.509736 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:12.509743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:12.509882 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.509995 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.510166 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
I20260619 20:48:12.510919 31330 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40631
I20260619 20:48:12.511042 31330 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:12.511255 31330 heartbeater.cc:511] Master 127.13.195.126:40631 requested a full tablet report, sending...
I20260619 20:48:12.511809 31139 ts_manager.cc:194] Registered new tserver with Master: 93644b508d5a44f199afb295d921dca6 (127.13.195.65:39323)
I20260619 20:48:12.512370 31139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:45303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:48:12.527632 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40631
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:12.610348 31335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:12.610569 31335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:12.610592 31335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:12.611749 31335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:12.611792 31335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:12.611851 31335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:48:12.613332 31335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:12.613531 31335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:12.613711 31335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:12.618731 31340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:12.618738 31341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:12.618747 31335 server_base.cc:1034] running on GCE node
W20260619 20:48:12.618731 31343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:12.619197 31335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:12.619439 31335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:12.620587 31335 hybrid_clock.cc:648] HybridClock initialized: now 1781902092620574 us; error 28 us; skew 500 ppm
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:12.623085 31335 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:48:12.623456 31335 webserver.cc:460] Webserver started at http://127.13.195.66:41121/ using document root <none> and password file <none>
I20260619 20:48:12.623697 31335 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:12.623744 31335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:12.623951 31335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:12.625006 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "71920b3aa766478a905531520afd5270"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "ef588f11716acf7d538ce5836d68d9c8"
server_key_iv: "e0e509eeafa7246df7398190fe13b3c1"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.625367 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "71920b3aa766478a905531520afd5270"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "ef588f11716acf7d538ce5836d68d9c8"
server_key_iv: "e0e509eeafa7246df7398190fe13b3c1"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.626767 31335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:48:12.627506 31350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.627699 31335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:12.627784 31335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "71920b3aa766478a905531520afd5270"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "ef588f11716acf7d538ce5836d68d9c8"
server_key_iv: "e0e509eeafa7246df7398190fe13b3c1"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.627837 31335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:12.634373 31335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:12.636189 31335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:12.636297 31335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:12.636473 31335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:12.636713 31335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:12.636744 31335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.636765 31335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:12.636776 31335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.642596 31335 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:37545
I20260619 20:48:12.642686 31463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:37545 every 8 connection(s)
I20260619 20:48:12.643007 31335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:12.651743 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.643735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:51123 (local address 127.13.195.126:40631)
0619 20:48:12.643814 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.643816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.643984 (+ 168us) server_negotiation.cc:407] Connection header received
0619 20:48:12.644432 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.644433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.644443 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.644496 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:12.645397 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.645902 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.646995 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.647183 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.647243 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.649294 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:12.649296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:12.649296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:12.649303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:12.650610 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.650976 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.650977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.650978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.651027 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.651236 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.651236 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.651237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.651337 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:12.651386 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.651614 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.651677 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:48:12.651959 31466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.643688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40631 (local address 127.13.195.66:51123)
0619 20:48:12.643854 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.643890 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.644353 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.644548 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.644551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.645128 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:12.645319 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.645323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.645984 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.645984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.646911 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.646913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.647257 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.647257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.647358 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.647861 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:12.647869 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:12.649208 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:12.650680 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:12.650686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:12.650687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:12.650903 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:12.651085 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:12.651086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:12.651086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:12.651182 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:12.651390 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:12.651394 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:12.651506 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.651586 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.651709 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260619 20:48:12.652549 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31335
I20260619 20:48:12.652550 31464 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40631
I20260619 20:48:12.652663 31464 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:12.652678 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:48:12.652855 31464 heartbeater.cc:511] Master 127.13.195.126:40631 requested a full tablet report, sending...
I20260619 20:48:12.652890 14093 external_mini_cluster.cc:1428] Setting key c572a53b5b40e55779a6cfa94742f3e2
I20260619 20:48:12.653319 31138 ts_manager.cc:194] Registered new tserver with Master: 71920b3aa766478a905531520afd5270 (127.13.195.66:37545)
I20260619 20:48:12.653790 31138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:51123
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:48:12.677345 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40631
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:12.757351 31469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:12.757555 31469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:12.757578 31469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:12.758731 31469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:12.758770 31469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:12.758796 31469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:48:12.760169 31469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:12.760349 31469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:12.760519 31469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:12.765395 31474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:12.765451 31475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:12.765532 31469 server_base.cc:1034] running on GCE node
W20260619 20:48:12.765420 31477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:12.765872 31469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:12.766175 31469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:12.767364 31469 hybrid_clock.cc:648] HybridClock initialized: now 1781902092767344 us; error 35 us; skew 500 ppm
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:12.769743 31469 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:48:12.770208 31469 webserver.cc:460] Webserver started at http://127.13.195.67:36065/ using document root <none> and password file <none>
I20260619 20:48:12.770479 31469 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:12.770555 31469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:12.770718 31469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:12.771826 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "c1d2dbe271284916b7c597d3a10b69be"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "45178f3a070f6de17fcfec47a7276530"
server_key_iv: "5d494db81abfd83681aa697bbc699d51"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.772187 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "c1d2dbe271284916b7c597d3a10b69be"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "45178f3a070f6de17fcfec47a7276530"
server_key_iv: "5d494db81abfd83681aa697bbc699d51"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.773588 31469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:12.774308 31484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.774502 31469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:12.774598 31469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "c1d2dbe271284916b7c597d3a10b69be"
format_stamp: "Formatted at 2026-06-19 20:48:12 on dist-test-slave-n2td"
server_key: "45178f3a070f6de17fcfec47a7276530"
server_key_iv: "5d494db81abfd83681aa697bbc699d51"
server_key_version: "encryptionkey@0"
I20260619 20:48:12.774691 31469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:12.792639 31469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:12.794509 31469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:12.794672 31469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:12.794898 31469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:12.795212 31469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:12.795269 31469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.795310 31469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:12.795336 31469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:12.801054 31469 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:35229
I20260619 20:48:12.801096 31597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:35229 every 8 connection(s)
I20260619 20:48:12.801402 31469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:48:12.802034 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31469
I20260619 20:48:12.802150 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:48:12.802366 14093 external_mini_cluster.cc:1428] Setting key 6f3da5102d2547cb55e5c66d8d0d4f1a
Jun 19 20:48:12 dist-test-slave-n2td krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902092, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:12.810115 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.802109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:44763 (local address 127.13.195.126:40631)
0619 20:48:12.802165 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.802167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.802436 (+ 269us) server_negotiation.cc:407] Connection header received
0619 20:48:12.803108 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.803109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.803120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.803165 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:12.804148 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.804649 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.805581 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.805769 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.805819 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.807821 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:12.807822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:12.807822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:12.807828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:12.809022 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.809364 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.809365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.809365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.809414 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.809626 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.809626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.809626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.809725 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:12.809775 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.809993 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.810051 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:48:12.810289 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.802067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40631 (local address 127.13.195.67:44763)
0619 20:48:12.802270 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.802322 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.802994 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.803237 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.803241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.803888 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:12.804071 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.804075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.804723 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.804724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.805510 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.805511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.805838 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.805838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.805933 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.806349 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:12.806357 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:12.807742 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:12.809090 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:12.809096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:12.809098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:12.809299 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:12.809471 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:12.809472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:12.809472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:12.809562 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:12.809778 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:12.809782 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:12.809887 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.809965 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.810087 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260619 20:48:12.810909 31598 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40631
I20260619 20:48:12.811002 31598 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:12.811158 31598 heartbeater.cc:511] Master 127.13.195.126:40631 requested a full tablet report, sending...
I20260619 20:48:12.811631 31138 ts_manager.cc:194] Registered new tserver with Master: c1d2dbe271284916b7c597d3a10b69be (127.13.195.67:35229)
I20260619 20:48:12.812078 31138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:44763
I20260619 20:48:12.814175 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:48:12.821653 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.815539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.13.195.126:40631)
0619 20:48:12.815601 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.815603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.815691 (+ 88us) server_negotiation.cc:407] Connection header received
0619 20:48:12.815745 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.815745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.815753 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.815811 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:12.816302 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.816756 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.817424 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.817592 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.817650 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.818907 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:12.818908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:12.818908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:12.818914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:12.820289 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.820766 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.820768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.820769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.820860 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:12.821082 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:12.821083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:12.821083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:12.821246 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:12.821353 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.821457 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.821535 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:48:12.823459 31138 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:48:12.825110 31138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:48:12.831959 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.827556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39323 (local address 127.0.0.1:57262)
0619 20:48:12.827773 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.827782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.827825 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.828094 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.828095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.828097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:12.828238 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.828241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.829577 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.829578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.830730 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.830732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.831652 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.831653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.831731 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.831746 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.831824 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.831875 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":82,"thread_start_us":85,"threads_started":1}
I20260619 20:48:12.831995 31614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.827762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.13.195.65:39323)
0619 20:48:12.827916 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.827920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.827931 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:48:12.827972 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.827974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.827990 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.828133 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:12.828356 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.829487 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.830806 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.831544 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.831648 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.831710 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.831849 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.831890 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260619 20:48:12.832887 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.827361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:37545 (local address 127.0.0.1:54880)
0619 20:48:12.827472 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.827515 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.827586 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.827973 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.827976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.827979 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:12.828325 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.828326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.830528 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.830529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.831347 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.831350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.832672 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.832673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.832733 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.832737 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.832792 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.832836 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":31,"thread_start_us":60,"threads_started":1}
I20260619 20:48:12.832808 31264 tablet_service.cc:1467] Processing CreateTablet for tablet b5de387efeec442292a55b0f4bc9ca3d (DEFAULT_TABLE table=test-table [id=d356f326a9144519965f7e645f6029cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:12.833144 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.827445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54880 (local address 127.13.195.66:37545)
0619 20:48:12.827800 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.827805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.827814 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:48:12.827857 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.827859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.827877 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.827951 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:12.828379 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.829475 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.831452 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.832580 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.832659 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.832712 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.832822 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.832866 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":88,"threads_started":1}
I20260619 20:48:12.833209 31264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5de387efeec442292a55b0f4bc9ca3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:12.833221 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.827366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:35229 (local address 127.0.0.1:55606)
0619 20:48:12.827621 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.827631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.827681 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.828183 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.828184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.828187 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:12.828243 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.828246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.830677 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.830678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.831885 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.831888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.832996 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.832996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.833059 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.833062 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.833124 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.833162 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
I20260619 20:48:12.833264 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.827521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.13.195.67:35229)
0619 20:48:12.827949 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.827953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.827962 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:48:12.827998 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.828000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.828014 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.828083 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:12.829615 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.830559 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.831973 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.832578 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.832678 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.832734 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.833143 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.833183 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":80,"threads_started":1}
I20260619 20:48:12.833890 31532 tablet_service.cc:1467] Processing CreateTablet for tablet b5de387efeec442292a55b0f4bc9ca3d (DEFAULT_TABLE table=test-table [id=d356f326a9144519965f7e645f6029cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:12.834137 31532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5de387efeec442292a55b0f4bc9ca3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:12.834156 31398 tablet_service.cc:1467] Processing CreateTablet for tablet b5de387efeec442292a55b0f4bc9ca3d (DEFAULT_TABLE table=test-table [id=d356f326a9144519965f7e645f6029cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:12.834501 31398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5de387efeec442292a55b0f4bc9ca3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:12.836676 31615 tablet_bootstrap.cc:492] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Bootstrap starting.
I20260619 20:48:12.837268 31615 tablet_bootstrap.cc:654] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:12.837329 31616 tablet_bootstrap.cc:492] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6: Bootstrap starting.
I20260619 20:48:12.837664 31615 log.cc:824] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:12.838039 31616 tablet_bootstrap.cc:654] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:12.838418 31615 tablet_bootstrap.cc:492] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: No bootstrap required, opened a new log
I20260619 20:48:12.838547 31615 ts_tablet_manager.cc:1397] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:48:12.838603 31617 tablet_bootstrap.cc:492] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Bootstrap starting.
I20260619 20:48:12.838619 31616 log.cc:824] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:12.839233 31617 tablet_bootstrap.cc:654] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:12.839399 31616 tablet_bootstrap.cc:492] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6: No bootstrap required, opened a new log
I20260619 20:48:12.839473 31616 ts_tablet_manager.cc:1397] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:48:12.839840 31617 log.cc:824] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:12.840466 31615 raft_consensus.cc:348] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.840618 31617 tablet_bootstrap.cc:492] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: No bootstrap required, opened a new log
I20260619 20:48:12.840636 31615 raft_consensus.cc:374] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:12.840677 31617 ts_tablet_manager.cc:1397] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:48:12.840679 31615 raft_consensus.cc:729] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d2dbe271284916b7c597d3a10b69be, State: Initialized, Role: FOLLOWER
I20260619 20:48:12.840811 31615 consensus_queue.cc:260] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.841094 31615 ts_tablet_manager.cc:1428] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 20:48:12.841221 31598 heartbeater.cc:503] Master 127.13.195.126:40631 was elected leader, sending a full tablet report...
I20260619 20:48:12.841863 31616 raft_consensus.cc:348] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.842044 31616 raft_consensus.cc:374] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:12.842082 31616 raft_consensus.cc:729] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93644b508d5a44f199afb295d921dca6, State: Initialized, Role: FOLLOWER
I20260619 20:48:12.842195 31616 consensus_queue.cc:260] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.842520 31330 heartbeater.cc:503] Master 127.13.195.126:40631 was elected leader, sending a full tablet report...
I20260619 20:48:12.842478 31616 ts_tablet_manager.cc:1428] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:48:12.842693 31617 raft_consensus.cc:348] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.842841 31617 raft_consensus.cc:374] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:12.842883 31617 raft_consensus.cc:729] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71920b3aa766478a905531520afd5270, State: Initialized, Role: FOLLOWER
I20260619 20:48:12.843003 31617 consensus_queue.cc:260] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.843278 31617 ts_tablet_manager.cc:1428] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:48:12.843394 31464 heartbeater.cc:503] Master 127.13.195.126:40631 was elected leader, sending a full tablet report...
I20260619 20:48:12.872085 31621 raft_consensus.cc:482] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:12.872212 31621 raft_consensus.cc:504] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.872611 31621 leader_election.cc:288] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71920b3aa766478a905531520afd5270 (127.13.195.66:37545), 93644b508d5a44f199afb295d921dca6 (127.13.195.65:39323)
I20260619 20:48:12.875414 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.872854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:58363 (local address 127.13.195.66:37545)
0619 20:48:12.872914 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.872916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.872926 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:12.872958 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.872959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.872969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.873037 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:12.873215 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.873697 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.874666 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.875045 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.875147 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.875156 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.875311 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.875346 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:48:12.875415 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.872750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:37545 (local address 127.13.195.67:58363)
0619 20:48:12.872814 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.872826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.872870 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.873056 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.873057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.873059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:12.873138 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.873139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.873784 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.873785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.874577 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.874580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.875149 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.875149 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.875236 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.875239 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.875306 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.875346 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 20:48:12.875731 31614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.872949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:37589 (local address 127.13.195.65:39323)
0619 20:48:12.873019 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.873021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.873157 (+ 136us) server_negotiation.cc:407] Connection header received
0619 20:48:12.873234 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.873235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.873245 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.873313 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:12.873476 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.873927 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.874901 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.875278 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.875363 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.875371 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.875628 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.875669 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:48:12.875777 31418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5de387efeec442292a55b0f4bc9ca3d" candidate_uuid: "c1d2dbe271284916b7c597d3a10b69be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71920b3aa766478a905531520afd5270" is_pre_election: true
I20260619 20:48:12.875929 31418 raft_consensus.cc:2454] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d2dbe271284916b7c597d3a10b69be in term 0.
I20260619 20:48:12.876169 31624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.872870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39323 (local address 127.13.195.67:37589)
0619 20:48:12.873090 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.873100 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:12.873169 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:12.873318 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:12.873319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:12.873320 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:12.873400 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.873401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.874011 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.874012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.874815 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:12.874817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.875361 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.875362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:12.875437 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.875440 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:12.875512 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:48:12.875558 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":146,"threads_started":1}
I20260619 20:48:12.876190 31486 leader_election.cc:302] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71920b3aa766478a905531520afd5270, c1d2dbe271284916b7c597d3a10b69be; no voters:
I20260619 20:48:12.876346 31621 raft_consensus.cc:2781] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:48:12.876430 31621 raft_consensus.cc:482] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:48:12.876467 31621 raft_consensus.cc:3037] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:12.876587 31284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5de387efeec442292a55b0f4bc9ca3d" candidate_uuid: "c1d2dbe271284916b7c597d3a10b69be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93644b508d5a44f199afb295d921dca6" is_pre_election: true
I20260619 20:48:12.876744 31284 raft_consensus.cc:2454] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1d2dbe271284916b7c597d3a10b69be in term 0.
I20260619 20:48:12.877424 31621 raft_consensus.cc:504] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.877553 31621 leader_election.cc:288] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [CANDIDATE]: Term 1 election: Requested vote from peers 71920b3aa766478a905531520afd5270 (127.13.195.66:37545), 93644b508d5a44f199afb295d921dca6 (127.13.195.65:39323)
I20260619 20:48:12.877671 31418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5de387efeec442292a55b0f4bc9ca3d" candidate_uuid: "c1d2dbe271284916b7c597d3a10b69be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71920b3aa766478a905531520afd5270"
I20260619 20:48:12.877704 31284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5de387efeec442292a55b0f4bc9ca3d" candidate_uuid: "c1d2dbe271284916b7c597d3a10b69be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93644b508d5a44f199afb295d921dca6"
I20260619 20:48:12.877746 31418 raft_consensus.cc:3037] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:12.877780 31284 raft_consensus.cc:3037] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:12.878566 31284 raft_consensus.cc:2454] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d2dbe271284916b7c597d3a10b69be in term 1.
I20260619 20:48:12.878631 31418 raft_consensus.cc:2454] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1d2dbe271284916b7c597d3a10b69be in term 1.
I20260619 20:48:12.878736 31486 leader_election.cc:302] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93644b508d5a44f199afb295d921dca6, c1d2dbe271284916b7c597d3a10b69be; no voters:
I20260619 20:48:12.878839 31621 raft_consensus.cc:2781] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:12.878981 31621 raft_consensus.cc:686] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 1 LEADER]: Becoming Leader. State: Replica: c1d2dbe271284916b7c597d3a10b69be, State: Running, Role: LEADER
I20260619 20:48:12.879078 31621 consensus_queue.cc:237] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } }
I20260619 20:48:12.879853 31139 catalog_manager.cc:5495] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be reported cstate change: term changed from 0 to 1, leader changed from <none> to c1d2dbe271284916b7c597d3a10b69be (127.13.195.67). New cstate: current_term: 1 leader_uuid: "c1d2dbe271284916b7c597d3a10b69be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d2dbe271284916b7c597d3a10b69be" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 35229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 } health_report { overall_health: UNKNOWN } } }
I20260619 20:48:12.893504 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.891018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.13.195.67:35229)
0619 20:48:12.891078 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:12.891081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:12.891093 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:48:12.891205 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:12.891206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:12.891231 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:12.891307 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:12.891439 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.891916 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.892694 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:12.892894 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:12.892967 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:12.893142 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:12.893253 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:12.893353 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:12.893370 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 20:48:12.893390 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 20:48:12.893594 31465 tablet.cc:2369] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:48:12.895360 31418 raft_consensus.cc:1261] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 1 FOLLOWER]: Refusing update from remote peer c1d2dbe271284916b7c597d3a10b69be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:12.895669 31284 raft_consensus.cc:1261] T b5de387efeec442292a55b0f4bc9ca3d P 93644b508d5a44f199afb295d921dca6 [term 1 FOLLOWER]: Refusing update from remote peer c1d2dbe271284916b7c597d3a10b69be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:12.895707 31625 consensus_queue.cc:1035] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [LEADER]: Connected to new peer: Peer: permanent_uuid: "71920b3aa766478a905531520afd5270" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 37545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:12.895926 31625 consensus_queue.cc:1035] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [LEADER]: Connected to new peer: Peer: permanent_uuid: "93644b508d5a44f199afb295d921dca6" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 39323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:12.898032 31628 mvcc.cc:205] Tried to move back new op lower bound from 7298670972496773120 to 7298670972433313792. Current Snapshot: MvccSnapshot[applied={T|T < 7298670972496773120}]
I20260619 20:48:12.900305 31139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:12.900404 31139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:12.901139 31139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e71125030f3d4584b4a846e1e0f3a622: Sending DeleteTablet for 3 replicas of tablet b5de387efeec442292a55b0f4bc9ca3d
I20260619 20:48:12.901525 31532 tablet_service.cc:1514] Processing DeleteTablet for tablet b5de387efeec442292a55b0f4bc9ca3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:12 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:55606
I20260619 20:48:12.901538 31398 tablet_service.cc:1514] Processing DeleteTablet for tablet b5de387efeec442292a55b0f4bc9ca3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:12 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:54880
I20260619 20:48:12.901568 31264 tablet_service.cc:1514] Processing DeleteTablet for tablet b5de387efeec442292a55b0f4bc9ca3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:12 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:57262
I20260619 20:48:12.901744 31639 tablet_replica.cc:330] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: stopping tablet replica
I20260619 20:48:12.901832 31639 raft_consensus.cc:2229] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:12.901930 31639 raft_consensus.cc:2258] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:12.902084 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31201
I20260619 20:48:12.902120 31638 tablet_replica.cc:330] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: stopping tablet replica
I20260619 20:48:12.902209 31638 raft_consensus.cc:2229] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:48:12.902307 31638 raft_consensus.cc:2258] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:12.902804 31638 ts_tablet_manager.cc:1905] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:12.903085 31639 ts_tablet_manager.cc:1905] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:12.904526 31638 ts_tablet_manager.cc:1918] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:12.904606 31638 log.cc:1196] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/b5de387efeec442292a55b0f4bc9ca3d
I20260619 20:48:12.904901 31638 ts_tablet_manager.cc:1939] T b5de387efeec442292a55b0f4bc9ca3d P c1d2dbe271284916b7c597d3a10b69be: Deleting consensus metadata
I20260619 20:48:12.905071 31639 ts_tablet_manager.cc:1918] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:12.905187 31639 log.cc:1196] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/b5de387efeec442292a55b0f4bc9ca3d
I20260619 20:48:12.905550 31639 ts_tablet_manager.cc:1939] T b5de387efeec442292a55b0f4bc9ca3d P 71920b3aa766478a905531520afd5270: Deleting consensus metadata
I20260619 20:48:12.906211 31124 catalog_manager.cc:4841] TS 71920b3aa766478a905531520afd5270 (127.13.195.66:37545): tablet b5de387efeec442292a55b0f4bc9ca3d (table test-table [id=d356f326a9144519965f7e645f6029cc]) successfully deleted
I20260619 20:48:12.906469 31126 catalog_manager.cc:4841] TS c1d2dbe271284916b7c597d3a10b69be (127.13.195.67:35229): tablet b5de387efeec442292a55b0f4bc9ca3d (table test-table [id=d356f326a9144519965f7e645f6029cc]) successfully deleted
W20260619 20:48:12.910324 31124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:39323 (error 108)
I20260619 20:48:12.910593 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31335
I20260619 20:48:12.911345 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:12.910739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:39323 (local address 127.0.0.1:57272)
0619 20:48:12.911265 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:12.911304 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:39323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508}
W20260619 20:48:12.912557 31124 catalog_manager.cc:4584] TS 93644b508d5a44f199afb295d921dca6 (127.13.195.65:39323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b5de387efeec442292a55b0f4bc9ca3d: Network error: Client connection negotiation failed: client connection to 127.13.195.65:39323: connect: Connection refused (error 111)
I20260619 20:48:12.915771 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31469
I20260619 20:48:12.920590 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31107
2026-06-19T20:48:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1932 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T20:48:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:12Z Disabled control of system clock
I20260619 20:48:12.948328 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:35369
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:35369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:13.028213 31645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:13.028431 31645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:13.028452 31645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:13.029543 31645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:13.029578 31645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:13.029592 31645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:13.029603 31645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:13.029623 31645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:13.030903 31645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:35369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:35369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:13.031100 31645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:13.031342 31645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:13.036283 31650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:13.036288 31653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:13.036319 31651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:13.036759 31645 server_base.cc:1034] running on GCE node
I20260619 20:48:13.037035 31645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:13.037293 31645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:13.038486 31645 hybrid_clock.cc:648] HybridClock initialized: now 1781902093038464 us; error 38 us; skew 500 ppm
I20260619 20:48:13.039701 31645 webserver.cc:460] Webserver started at http://127.13.195.126:36113/ using document root <none> and password file <none>
I20260619 20:48:13.039917 31645 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:13.039958 31645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:13.040095 31645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:13.041138 31645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "518f8553a77941eeaedf91ecfea97bcd"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "6b12963148444512f7bf2ccb52bcef03"
server_key_iv: "81470ed8a7711e703a602aa367db7807"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.041479 31645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "518f8553a77941eeaedf91ecfea97bcd"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "6b12963148444512f7bf2ccb52bcef03"
server_key_iv: "81470ed8a7711e703a602aa367db7807"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.042860 31645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:13.043574 31659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.043748 31645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:13.043849 31645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "518f8553a77941eeaedf91ecfea97bcd"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "6b12963148444512f7bf2ccb52bcef03"
server_key_iv: "81470ed8a7711e703a602aa367db7807"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.043922 31645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:13.058756 31645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:13.059043 31645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:13.059188 31645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:13.063336 31645 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:35369
I20260619 20:48:13.063401 31711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:35369 every 8 connection(s)
I20260619 20:48:13.063704 31645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:13.064608 31712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:13.066951 31712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd: Bootstrap starting.
I20260619 20:48:13.067517 31712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:13.067826 31712 log.cc:824] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:13.068506 31712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd: No bootstrap required, opened a new log
I20260619 20:48:13.070405 31712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518f8553a77941eeaedf91ecfea97bcd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35369 } }
I20260619 20:48:13.070632 31712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:13.070688 31712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518f8553a77941eeaedf91ecfea97bcd, State: Initialized, Role: FOLLOWER
I20260619 20:48:13.070828 31712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518f8553a77941eeaedf91ecfea97bcd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35369 } }
I20260619 20:48:13.070911 31712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:13.070950 31712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:13.070998 31712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:13.071497 31712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518f8553a77941eeaedf91ecfea97bcd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35369 } }
I20260619 20:48:13.071620 31712 leader_election.cc:302] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 518f8553a77941eeaedf91ecfea97bcd; no voters:
I20260619 20:48:13.071805 31712 leader_election.cc:288] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:13.071838 31715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:13.072021 31715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [term 1 LEADER]: Becoming Leader. State: Replica: 518f8553a77941eeaedf91ecfea97bcd, State: Running, Role: LEADER
I20260619 20:48:13.072058 31712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:13.072121 31715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518f8553a77941eeaedf91ecfea97bcd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35369 } }
I20260619 20:48:13.072547 31717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 518f8553a77941eeaedf91ecfea97bcd. Latest consensus state: current_term: 1 leader_uuid: "518f8553a77941eeaedf91ecfea97bcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518f8553a77941eeaedf91ecfea97bcd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35369 } } }
I20260619 20:48:13.072649 31717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:13.072697 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31645
I20260619 20:48:13.072677 31716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "518f8553a77941eeaedf91ecfea97bcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518f8553a77941eeaedf91ecfea97bcd" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35369 } } }
I20260619 20:48:13.072816 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:13.072813 31716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:13.072883 31724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:13.073035 14093 external_mini_cluster.cc:1428] Setting key 4138bc1b626e6f38dd9506e17896c529
I20260619 20:48:13.073387 31724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:13.075417 31724 catalog_manager.cc:1347] Generated new cluster ID: 671a8dab545b400cae632dfebea8b221
I20260619 20:48:13.075471 31724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:13.078759 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.073564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.13.195.126:35369)
0619 20:48:13.073923 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.073933 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.073968 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:48:13.074141 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.074151 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.074232 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.074356 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:13.074644 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.075878 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.076759 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.077178 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.077310 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.077996 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:13.078000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:13.078000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:13.078196 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:13.078231 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:13.078242 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:13.078320 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.078395 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.078607 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":66,"threads_started":1}
I20260619 20:48:13.093317 31724 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:13.093852 31724 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:13.097687 31724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd: Generated new TSK 0
I20260619 20:48:13.097877 31724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:13.099301 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:35369
--builtin_ntp_servers=127.13.195.84:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:13.203977 31736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:13.204268 31736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:13.204309 31736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:13.205487 31736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:13.205550 31736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:13.205591 31736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:48:13.206943 31736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:35369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:13.207203 31736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:13.207492 31736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:13.213435 31744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:13.213454 31742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:13.213435 31741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:13.213652 31736 server_base.cc:1034] running on GCE node
I20260619 20:48:13.213881 31736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:13.214159 31736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:13.215349 31736 hybrid_clock.cc:648] HybridClock initialized: now 1781902093215326 us; error 35 us; skew 500 ppm
I20260619 20:48:13.216878 31736 webserver.cc:460] Webserver started at http://127.13.195.65:44245/ using document root <none> and password file <none>
I20260619 20:48:13.217146 31736 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:13.217193 31736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:13.217324 31736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:13.218418 31736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "80b3d1c2198b4f13938bafa7ed9ce037"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "da5866cce04d99eaa1b2f5b5501d583e"
server_key_iv: "56f491c0c1fb42fb1e0f1b95f424e889"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.218927 31736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "80b3d1c2198b4f13938bafa7ed9ce037"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "da5866cce04d99eaa1b2f5b5501d583e"
server_key_iv: "56f491c0c1fb42fb1e0f1b95f424e889"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.220335 31736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:48:13.221109 31750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.221278 31736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:13.221387 31736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "80b3d1c2198b4f13938bafa7ed9ce037"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "da5866cce04d99eaa1b2f5b5501d583e"
server_key_iv: "56f491c0c1fb42fb1e0f1b95f424e889"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.221468 31736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:13.235875 31736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:13.236212 31736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:13.236370 31736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:13.236611 31736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:13.236963 31736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:13.237021 31736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.237057 31736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:13.237083 31736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.243268 31736 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:38619
I20260619 20:48:13.243311 31863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:38619 every 8 connection(s)
I20260619 20:48:13.243686 31736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:48:13.244861 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31736
I20260619 20:48:13.244973 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:48:13.245210 14093 external_mini_cluster.cc:1428] Setting key f0724ce6ca67b3c08b98df9f7a377214
I20260619 20:48:13.246342 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:35369
--builtin_ntp_servers=127.13.195.84:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 20:48:13.249964 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.244398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:55501 (local address 127.13.195.126:35369)
0619 20:48:13.244484 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.244487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.244750 (+ 263us) server_negotiation.cc:407] Connection header received
0619 20:48:13.245478 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.245480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.245497 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.245560 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:13.246516 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.247140 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.248445 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.248649 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.248738 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.249652 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:13.249654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:13.249654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:13.249660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:13.249681 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:13.249685 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:13.249749 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.249849 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.249880 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 20:48:13.250195 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.244368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:35369 (local address 127.13.195.65:55501)
0619 20:48:13.244580 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.244623 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.245370 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.245652 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.245656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.245665 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:13.246372 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.246379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.247237 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.247238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.248332 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.248334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.248750 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.248751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.248897 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.249483 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:13.249483 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:13.249513 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:13.249537 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:13.249749 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:13.249756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:13.249759 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.249853 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.249960 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260619 20:48:13.250799 31864 heartbeater.cc:347] Connected to a master server at 127.13.195.126:35369
I20260619 20:48:13.250914 31864 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:13.251102 31864 heartbeater.cc:511] Master 127.13.195.126:35369 requested a full tablet report, sending...
I20260619 20:48:13.251715 31676 ts_manager.cc:194] Registered new tserver with Master: 80b3d1c2198b4f13938bafa7ed9ce037 (127.13.195.65:38619)
I20260619 20:48:13.252767 31676 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.65:55501
W20260619 20:48:13.329818 31867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:13.330019 31867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:13.330039 31867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:13.331120 31867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:13.331154 31867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:13.331179 31867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:48:13.332417 31867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:35369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:13.332695 31867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:13.332990 31867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:13.338013 31875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:13.338021 31872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:13.338042 31873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:13.338485 31867 server_base.cc:1034] running on GCE node
I20260619 20:48:13.338688 31867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:13.338877 31867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:13.340022 31867 hybrid_clock.cc:648] HybridClock initialized: now 1781902093340012 us; error 32 us; skew 500 ppm
I20260619 20:48:13.341248 31867 webserver.cc:460] Webserver started at http://127.13.195.66:46391/ using document root <none> and password file <none>
I20260619 20:48:13.341439 31867 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:13.341478 31867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:13.341566 31867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:13.342654 31867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "a2f95a9829ab454eb14b3fabd8ffb51c"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "5c8e8f34ac4cb708f9f52e751db8b231"
server_key_iv: "36a24957f2a93082c098ce658b77153d"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.342950 31867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "a2f95a9829ab454eb14b3fabd8ffb51c"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "5c8e8f34ac4cb708f9f52e751db8b231"
server_key_iv: "36a24957f2a93082c098ce658b77153d"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.344106 31867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:13.344712 31881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.344872 31867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:13.344956 31867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "a2f95a9829ab454eb14b3fabd8ffb51c"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "5c8e8f34ac4cb708f9f52e751db8b231"
server_key_iv: "36a24957f2a93082c098ce658b77153d"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.345009 31867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:13.353972 31867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:13.354197 31867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:13.354316 31867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:13.354568 31867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:13.354835 31867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:13.354866 31867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.354887 31867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:13.354898 31867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.360368 31867 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:46333
I20260619 20:48:13.360460 31994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:46333 every 8 connection(s)
I20260619 20:48:13.360770 31867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:48:13.361552 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31867
I20260619 20:48:13.361652 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:48:13.361867 14093 external_mini_cluster.cc:1428] Setting key 76a4a51e86669d22d3df045f3792981b
I20260619 20:48:13.362843 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:35369
--builtin_ntp_servers=127.13.195.84:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 20:48:13.366199 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.361478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:34171 (local address 127.13.195.126:35369)
0619 20:48:13.361548 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.361551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.361762 (+ 211us) server_negotiation.cc:407] Connection header received
0619 20:48:13.362313 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.362314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.362322 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.362369 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:13.363105 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.363868 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.364817 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.365016 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.365089 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.365901 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:13.365902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:13.365903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:13.365907 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:13.365922 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:13.365926 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:13.365990 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.366087 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.366121 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:48:13.366463 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.361440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:35369 (local address 127.13.195.66:34171)
0619 20:48:13.361647 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.361686 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.362198 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.362461 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.362466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.362477 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:13.363020 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.363028 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.363957 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.363958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.364722 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.364724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.365107 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.365108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.365239 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.365752 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:13.365752 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:13.365783 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:13.365803 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:13.365991 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:13.365997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:13.366000 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.366088 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.366187 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260619 20:48:13.366863 31995 heartbeater.cc:347] Connected to a master server at 127.13.195.126:35369
I20260619 20:48:13.366950 31995 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:13.367139 31995 heartbeater.cc:511] Master 127.13.195.126:35369 requested a full tablet report, sending...
I20260619 20:48:13.367522 31676 ts_manager.cc:194] Registered new tserver with Master: a2f95a9829ab454eb14b3fabd8ffb51c (127.13.195.66:46333)
I20260619 20:48:13.367911 31676 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.66:34171
W20260619 20:48:13.444564 31998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:13.444800 31998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:13.444835 31998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:13.445994 31998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:13.446063 31998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:13.446105 31998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:48:13.447664 31998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:35369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:13.447921 31998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:13.448237 31998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:13.453308 32004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:13.453310 32003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:13.453377 31998 server_base.cc:1034] running on GCE node
W20260619 20:48:13.453316 32006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:13.453740 31998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:13.453935 31998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:13.455078 31998 hybrid_clock.cc:648] HybridClock initialized: now 1781902093455058 us; error 38 us; skew 500 ppm
I20260619 20:48:13.456362 31998 webserver.cc:460] Webserver started at http://127.13.195.67:46385/ using document root <none> and password file <none>
I20260619 20:48:13.456622 31998 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:13.456670 31998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:13.456840 31998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:13.457937 31998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "e280e51b89714c209972bbb2fb931698"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "d80832638cd0b03a0f18a0fcdb14f29a"
server_key_iv: "214da6221c181190b1eeb4238f30799b"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.458293 31998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "e280e51b89714c209972bbb2fb931698"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "d80832638cd0b03a0f18a0fcdb14f29a"
server_key_iv: "214da6221c181190b1eeb4238f30799b"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.459584 31998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:13.460247 32012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.460414 31998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:13.460513 31998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "e280e51b89714c209972bbb2fb931698"
format_stamp: "Formatted at 2026-06-19 20:48:13 on dist-test-slave-n2td"
server_key: "d80832638cd0b03a0f18a0fcdb14f29a"
server_key_iv: "214da6221c181190b1eeb4238f30799b"
server_key_version: "encryptionkey@0"
I20260619 20:48:13.460590 31998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:13.478596 31998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:13.478893 31998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:13.479039 31998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:13.479272 31998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:13.479590 31998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:13.479647 31998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.479688 31998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:13.479717 31998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:13.485438 31998 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:42177
I20260619 20:48:13.485486 32125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:42177 every 8 connection(s)
I20260619 20:48:13.485801 31998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:48:13.487941 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 31998
I20260619 20:48:13.488023 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:48:13.488229 14093 external_mini_cluster.cc:1428] Setting key f2221849a6fa9a1025328ad6f13ed8b0
I20260619 20:48:13.491285 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.486555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:48029 (local address 127.13.195.126:35369)
0619 20:48:13.486627 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.486629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.486889 (+ 260us) server_negotiation.cc:407] Connection header received
0619 20:48:13.487418 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.487420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.487432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.487511 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:13.488277 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.489025 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.489988 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.490177 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.490228 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.491070 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:13.491071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:13.491071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:13.491075 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:13.491085 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:13.491088 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:13.491137 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.491209 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.491229 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:48:13.491462 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.486555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:35369 (local address 127.13.195.67:48029)
0619 20:48:13.486763 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.486797 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.487311 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.487590 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.487595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.487617 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:13.488169 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.488186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.489131 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.489132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.489910 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.489912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.490247 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.490248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.490350 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.490945 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:13.490946 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:13.490974 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:13.490997 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:13.491140 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:13.491142 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:13.491144 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.491192 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.491264 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260619 20:48:13.491948 32126 heartbeater.cc:347] Connected to a master server at 127.13.195.126:35369
I20260619 20:48:13.492057 32126 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:13.492241 32126 heartbeater.cc:511] Master 127.13.195.126:35369 requested a full tablet report, sending...
I20260619 20:48:13.492585 31676 ts_manager.cc:194] Registered new tserver with Master: e280e51b89714c209972bbb2fb931698 (127.13.195.67:42177)
I20260619 20:48:13.492945 31676 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.67:48029
I20260619 20:48:13.499818 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:48:13.503779 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.501227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.13.195.126:35369)
0619 20:48:13.501288 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.501290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.501403 (+ 113us) server_negotiation.cc:407] Connection header received
0619 20:48:13.501442 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.501442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.501452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.501503 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:13.501636 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.502077 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.502828 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.502997 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.503049 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.503586 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:13.503587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:13.503587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:13.503591 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:13.503603 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:13.503606 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:13.503649 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.503707 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.503726 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:48:13.505156 31676 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:48:13.506724 31676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:48:13.512686 32134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.508749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38619 (local address 127.0.0.1:33978)
0619 20:48:13.508922 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.508961 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.509024 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.509479 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.509480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.509482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:13.509518 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.509520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.510458 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.510459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.511557 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.511560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.512390 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.512391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.512468 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.512482 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.512557 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.512610 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":83,"thread_start_us":63,"threads_started":1}
I20260619 20:48:13.512727 32139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.509110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.13.195.65:38619)
0619 20:48:13.509289 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.509293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.509301 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:13.509338 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.509340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.509353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.509465 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:13.509596 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.510335 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.511645 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.512306 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.512383 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.512473 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.512584 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.512628 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260619 20:48:13.513085 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.508749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:46333 (local address 127.0.0.1:39722)
0619 20:48:13.508922 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.508961 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.509024 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.509376 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.509379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.509382 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:13.509464 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.509466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.510414 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.510415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.511805 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.511807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.512822 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.512823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.512897 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.512900 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.512975 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.513016 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":92,"threads_started":1}
I20260619 20:48:13.513103 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.508935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39722 (local address 127.13.195.66:46333)
0619 20:48:13.509171 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.509175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.509183 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:13.509239 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.509241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.509258 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.509341 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:13.509565 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.510322 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.511892 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.512633 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.512700 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.512781 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.512990 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.513028 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260619 20:48:13.513739 31798 tablet_service.cc:1467] Processing CreateTablet for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac (DEFAULT_TABLE table=test-table [id=165567311355402594fd56e768cd0ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:13.513866 31929 tablet_service.cc:1467] Processing CreateTablet for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac (DEFAULT_TABLE table=test-table [id=165567311355402594fd56e768cd0ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:13.514137 31798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:13.514149 31929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:13.514241 32136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.508757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:42177 (local address 127.0.0.1:35930)
0619 20:48:13.509260 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.509269 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.509339 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.510711 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.510712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.510715 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:13.510792 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.510793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.512002 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.512003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.512917 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.512920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.513974 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.513974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.514039 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.514042 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.514112 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.514159 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":34,"thread_start_us":92,"threads_started":1}
I20260619 20:48:13.514469 32138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.508977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.13.195.67:42177)
0619 20:48:13.510521 (+ 1544us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.510524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.510534 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:13.510570 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.510576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.510600 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.510678 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:13.510868 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.511917 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.513004 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.513796 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.513857 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.513940 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.514130 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.514165 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1504,"thread_start_us":76,"threads_started":1}
I20260619 20:48:13.514992 32060 tablet_service.cc:1467] Processing CreateTablet for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac (DEFAULT_TABLE table=test-table [id=165567311355402594fd56e768cd0ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:13.515295 32060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:13.517791 32140 tablet_bootstrap.cc:492] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Bootstrap starting.
I20260619 20:48:13.517922 32142 tablet_bootstrap.cc:492] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Bootstrap starting.
I20260619 20:48:13.518395 32140 tablet_bootstrap.cc:654] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:13.518528 32142 tablet_bootstrap.cc:654] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:13.518764 32140 log.cc:824] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:13.518832 32142 log.cc:824] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:13.519054 32141 tablet_bootstrap.cc:492] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037: Bootstrap starting.
I20260619 20:48:13.519550 32142 tablet_bootstrap.cc:492] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: No bootstrap required, opened a new log
I20260619 20:48:13.519579 32141 tablet_bootstrap.cc:654] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:13.519645 32142 ts_tablet_manager.cc:1397] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:48:13.519858 32141 log.cc:824] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:13.520457 32140 tablet_bootstrap.cc:492] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: No bootstrap required, opened a new log
I20260619 20:48:13.520529 32140 ts_tablet_manager.cc:1397] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 20:48:13.520756 32141 tablet_bootstrap.cc:492] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037: No bootstrap required, opened a new log
I20260619 20:48:13.520825 32141 ts_tablet_manager.cc:1397] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:48:13.521725 32142 raft_consensus.cc:348] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.521883 32142 raft_consensus.cc:374] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:13.521925 32142 raft_consensus.cc:729] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e280e51b89714c209972bbb2fb931698, State: Initialized, Role: FOLLOWER
I20260619 20:48:13.522037 32142 consensus_queue.cc:260] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.522320 32142 ts_tablet_manager.cc:1428] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:48:13.522478 32126 heartbeater.cc:503] Master 127.13.195.126:35369 was elected leader, sending a full tablet report...
I20260619 20:48:13.523190 32140 raft_consensus.cc:348] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.523356 32140 raft_consensus.cc:374] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:13.523389 32140 raft_consensus.cc:729] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2f95a9829ab454eb14b3fabd8ffb51c, State: Initialized, Role: FOLLOWER
I20260619 20:48:13.523455 32141 raft_consensus.cc:348] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.523610 32141 raft_consensus.cc:374] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:13.523650 32141 raft_consensus.cc:729] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b3d1c2198b4f13938bafa7ed9ce037, State: Initialized, Role: FOLLOWER
I20260619 20:48:13.523553 32140 consensus_queue.cc:260] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.523759 32141 consensus_queue.cc:260] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.523857 32140 ts_tablet_manager.cc:1428] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260619 20:48:13.523917 31995 heartbeater.cc:503] Master 127.13.195.126:35369 was elected leader, sending a full tablet report...
I20260619 20:48:13.523989 32141 ts_tablet_manager.cc:1428] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:48:13.524106 31864 heartbeater.cc:503] Master 127.13.195.126:35369 was elected leader, sending a full tablet report...
W20260619 20:48:13.611281 31996 tablet.cc:2369] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:48:13.724517 32148 raft_consensus.cc:482] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:13.724690 32148 raft_consensus.cc:504] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.725133 32148 leader_election.cc:288] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2f95a9829ab454eb14b3fabd8ffb51c (127.13.195.66:46333), e280e51b89714c209972bbb2fb931698 (127.13.195.67:42177)
I20260619 20:48:13.728065 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.725479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:47669 (local address 127.13.195.66:46333)
0619 20:48:13.725556 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.725558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.725570 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:48:13.725612 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.725613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.725626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.725682 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:13.725868 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.726376 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.727344 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.727719 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.727772 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.727779 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.727943 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.727986 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:48:13.728077 32138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.725391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:43049 (local address 127.13.195.67:42177)
0619 20:48:13.725459 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.725462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.725643 (+ 181us) server_negotiation.cc:407] Connection header received
0619 20:48:13.725697 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.725698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.725707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.725766 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:13.725961 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.726401 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.727377 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.727730 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.727821 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.727829 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.727979 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.728019 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:48:13.728067 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.725319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:46333 (local address 127.13.195.65:47669)
0619 20:48:13.725384 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.725397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.725443 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.725703 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.725704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.725706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:13.725782 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.725785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.726463 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.726464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.727255 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.727257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.727797 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.727798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.727868 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.727873 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.727922 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.727981 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":2}
I20260619 20:48:13.728099 32149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.725342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:42177 (local address 127.13.195.65:43049)
0619 20:48:13.725541 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.725552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:13.725624 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:13.725776 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:13.725777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:13.725778 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:13.725881 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.725883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.726524 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.726525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.727268 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:13.727270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.727839 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.727840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:13.727891 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.727894 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:13.727963 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:48:13.727999 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260619 20:48:13.728466 31949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5dbfd3c02b4fb28a9abce1f4a5cbac" candidate_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" is_pre_election: true
I20260619 20:48:13.728475 32080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5dbfd3c02b4fb28a9abce1f4a5cbac" candidate_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e280e51b89714c209972bbb2fb931698" is_pre_election: true
I20260619 20:48:13.728623 31949 raft_consensus.cc:2454] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b3d1c2198b4f13938bafa7ed9ce037 in term 0.
I20260619 20:48:13.728637 32080 raft_consensus.cc:2454] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b3d1c2198b4f13938bafa7ed9ce037 in term 0.
I20260619 20:48:13.728839 31754 leader_election.cc:302] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80b3d1c2198b4f13938bafa7ed9ce037, e280e51b89714c209972bbb2fb931698; no voters:
I20260619 20:48:13.728961 32148 raft_consensus.cc:2781] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:48:13.729014 32148 raft_consensus.cc:482] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:48:13.729036 32148 raft_consensus.cc:3037] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:13.729727 32148 raft_consensus.cc:504] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.729867 32148 leader_election.cc:288] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [CANDIDATE]: Term 1 election: Requested vote from peers a2f95a9829ab454eb14b3fabd8ffb51c (127.13.195.66:46333), e280e51b89714c209972bbb2fb931698 (127.13.195.67:42177)
I20260619 20:48:13.730037 32080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5dbfd3c02b4fb28a9abce1f4a5cbac" candidate_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e280e51b89714c209972bbb2fb931698"
I20260619 20:48:13.730038 31949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5dbfd3c02b4fb28a9abce1f4a5cbac" candidate_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c"
I20260619 20:48:13.730129 32080 raft_consensus.cc:3037] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:13.730120 31949 raft_consensus.cc:3037] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:13.731043 31949 raft_consensus.cc:2454] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b3d1c2198b4f13938bafa7ed9ce037 in term 1.
I20260619 20:48:13.731043 32080 raft_consensus.cc:2454] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b3d1c2198b4f13938bafa7ed9ce037 in term 1.
I20260619 20:48:13.731223 31753 leader_election.cc:302] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80b3d1c2198b4f13938bafa7ed9ce037, a2f95a9829ab454eb14b3fabd8ffb51c; no voters:
I20260619 20:48:13.731330 32148 raft_consensus.cc:2781] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:13.731464 32148 raft_consensus.cc:686] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [term 1 LEADER]: Becoming Leader. State: Replica: 80b3d1c2198b4f13938bafa7ed9ce037, State: Running, Role: LEADER
I20260619 20:48:13.731583 32148 consensus_queue.cc:237] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } }
I20260619 20:48:13.732456 31675 catalog_manager.cc:5495] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80b3d1c2198b4f13938bafa7ed9ce037 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b3d1c2198b4f13938bafa7ed9ce037" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 } health_report { overall_health: UNKNOWN } } }
W20260619 20:48:13.736279 32127 tablet.cc:2369] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:48:13.737718 32139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.735552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33994 (local address 127.13.195.65:38619)
0619 20:48:13.735599 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:13.735601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:13.735609 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:13.735664 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:13.735664 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:13.735679 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:13.735734 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:13.735828 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.736262 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.737044 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:13.737225 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:13.737275 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:13.737399 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:13.737488 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:13.737547 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:13.737581 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 20:48:13.737599 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:48:13.739379 32080 raft_consensus.cc:1261] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 1 FOLLOWER]: Refusing update from remote peer 80b3d1c2198b4f13938bafa7ed9ce037: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:13.739413 31949 raft_consensus.cc:1261] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 1 FOLLOWER]: Refusing update from remote peer 80b3d1c2198b4f13938bafa7ed9ce037: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:13.739737 32150 consensus_queue.cc:1035] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e280e51b89714c209972bbb2fb931698" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:13.739866 32148 consensus_queue.cc:1035] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2f95a9829ab454eb14b3fabd8ffb51c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:13.742506 32153 mvcc.cc:205] Tried to move back new op lower bound from 7298670975954042880 to 7298670975925207040. Current Snapshot: MvccSnapshot[applied={T|T < 7298670975954042880}]
I20260619 20:48:13.744738 31675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:13.744829 31675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:13.745548 31675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 518f8553a77941eeaedf91ecfea97bcd: Sending DeleteTablet for 3 replicas of tablet db5dbfd3c02b4fb28a9abce1f4a5cbac
I20260619 20:48:13.745872 31798 tablet_service.cc:1514] Processing DeleteTablet for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:13 UTC) from {username='slave'} at 127.0.0.1:33978
I20260619 20:48:13.745875 31929 tablet_service.cc:1514] Processing DeleteTablet for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:13 UTC) from {username='slave'} at 127.0.0.1:39722
I20260619 20:48:13.746095 32163 tablet_replica.cc:330] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: stopping tablet replica
I20260619 20:48:13.746107 32162 tablet_replica.cc:330] T db5dbfd3c02b4fb28a9abce1f4a5cbac P 80b3d1c2198b4f13938bafa7ed9ce037: stopping tablet replica
I20260619 20:48:13.746212 32163 raft_consensus.cc:2229] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:13.746227 32060 tablet_service.cc:1514] Processing DeleteTablet for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:13 UTC) from {username='slave'} at 127.0.0.1:35930
I20260619 20:48:13.746354 32163 raft_consensus.cc:2258] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:13.746501 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31736
I20260619 20:48:13.747785 32163 ts_tablet_manager.cc:1905] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:13.749140 32164 tablet_replica.cc:330] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: stopping tablet replica
I20260619 20:48:13.749255 32164 raft_consensus.cc:2229] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:13.749349 32164 raft_consensus.cc:2258] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:13.749536 32163 ts_tablet_manager.cc:1918] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:13.749621 32163 log.cc:1196] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/db5dbfd3c02b4fb28a9abce1f4a5cbac
I20260619 20:48:13.749830 32164 ts_tablet_manager.cc:1905] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:13.749871 32163 ts_tablet_manager.cc:1939] T db5dbfd3c02b4fb28a9abce1f4a5cbac P a2f95a9829ab454eb14b3fabd8ffb51c: Deleting consensus metadata
I20260619 20:48:13.750567 31662 catalog_manager.cc:4841] TS a2f95a9829ab454eb14b3fabd8ffb51c (127.13.195.66:46333): tablet db5dbfd3c02b4fb28a9abce1f4a5cbac (table test-table [id=165567311355402594fd56e768cd0ea0]) successfully deleted
I20260619 20:48:13.751885 32164 ts_tablet_manager.cc:1918] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:13.751960 32164 log.cc:1196] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/db5dbfd3c02b4fb28a9abce1f4a5cbac
I20260619 20:48:13.752205 32164 ts_tablet_manager.cc:1939] T db5dbfd3c02b4fb28a9abce1f4a5cbac P e280e51b89714c209972bbb2fb931698: Deleting consensus metadata
I20260619 20:48:13.752595 31663 catalog_manager.cc:4841] TS e280e51b89714c209972bbb2fb931698 (127.13.195.67:42177): tablet db5dbfd3c02b4fb28a9abce1f4a5cbac (table test-table [id=165567311355402594fd56e768cd0ea0]) successfully deleted
W20260619 20:48:13.754027 31660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:38619 (error 108)
I20260619 20:48:13.754276 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31867
I20260619 20:48:13.754846 32136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:13.754347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38619 (local address 127.0.0.1:34006)
0619 20:48:13.754769 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:13.754812 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400}
W20260619 20:48:13.754992 31660 catalog_manager.cc:4584] TS 80b3d1c2198b4f13938bafa7ed9ce037 (127.13.195.65:38619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db5dbfd3c02b4fb28a9abce1f4a5cbac: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38619: connect: Connection refused (error 111)
I20260619 20:48:13.760486 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31998
I20260619 20:48:13.764758 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 31645
2026-06-19T20:48:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (843 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:13 dist-test-slave-n2td krb5kdc[32168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:13 dist-test-slave-n2td krb5kdc[32168](info): set up 2 sockets
Jun 19 20:48:13 dist-test-slave-n2td krb5kdc[32168](info): commencing operation
krb5kdc: starting...
W20260619 20:48:14.805195 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:14 dist-test-slave-n2td krb5kdc[32168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:14.925000 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45309
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:15.005358 32181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:15.005626 32181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:15.005666 32181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:15.006853 32181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:15.006925 32181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:15.006949 32181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:15.006978 32181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:15.007009 32181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:15.008502 32181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:15.008796 32181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:15.009025 32181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:15.013824 32187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.013790 32189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.013796 32186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:15.014098 32181 server_base.cc:1034] running on GCE node
I20260619 20:48:15.014338 32181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:15.014639 32181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:15.015784 32181 hybrid_clock.cc:648] HybridClock initialized: now 1781902095015768 us; error 30 us; skew 500 ppm
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902095, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:15.018107 32181 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:15.018596 32181 webserver.cc:460] Webserver started at http://127.13.195.126:36201/ using document root <none> and password file <none>
I20260619 20:48:15.018854 32181 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:15.018923 32181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:15.019081 32181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:15.020143 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "5223da4d29f14eefae63ff468981a704"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "5deff9427dffc6a055112842503142c3"
server_key_iv: "321b427b272bdf36db7b55c222de02a7"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.020500 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "5223da4d29f14eefae63ff468981a704"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "5deff9427dffc6a055112842503142c3"
server_key_iv: "321b427b272bdf36db7b55c222de02a7"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.021824 32181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:15.022517 32196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.022689 32181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:15.022789 32181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "5223da4d29f14eefae63ff468981a704"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "5deff9427dffc6a055112842503142c3"
server_key_iv: "321b427b272bdf36db7b55c222de02a7"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.022876 32181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:15.034679 32181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:15.036490 32181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:15.036662 32181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:15.040532 32181 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:45309
I20260619 20:48:15.040567 32248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:45309 every 8 connection(s)
I20260619 20:48:15.040923 32181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:15.041814 32249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:15.044123 32249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704: Bootstrap starting.
I20260619 20:48:15.044705 32249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:15.044991 32249 log.cc:824] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:15.045684 32249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704: No bootstrap required, opened a new log
I20260619 20:48:15.047344 32249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5223da4d29f14eefae63ff468981a704" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45309 } }
I20260619 20:48:15.047479 32249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:15.047528 32249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5223da4d29f14eefae63ff468981a704, State: Initialized, Role: FOLLOWER
I20260619 20:48:15.047680 32249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5223da4d29f14eefae63ff468981a704" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45309 } }
I20260619 20:48:15.047742 32249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:15.047765 32249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:15.047832 32249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:15.048349 32249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5223da4d29f14eefae63ff468981a704" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45309 } }
I20260619 20:48:15.048456 32249 leader_election.cc:302] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5223da4d29f14eefae63ff468981a704; no voters:
I20260619 20:48:15.048646 32249 leader_election.cc:288] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:15.048699 32252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:15.048874 32252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [term 1 LEADER]: Becoming Leader. State: Replica: 5223da4d29f14eefae63ff468981a704, State: Running, Role: LEADER
I20260619 20:48:15.048883 32249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:15.049019 32252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5223da4d29f14eefae63ff468981a704" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45309 } }
I20260619 20:48:15.049734 32253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5223da4d29f14eefae63ff468981a704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5223da4d29f14eefae63ff468981a704" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45309 } } }
I20260619 20:48:15.049870 32253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:15.049959 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 32181
I20260619 20:48:15.050104 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:15.050143 32262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:15.050330 14093 external_mini_cluster.cc:1428] Setting key 77c5d36857d5ec8a7f3b02687a1b68e9
I20260619 20:48:15.050635 32254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5223da4d29f14eefae63ff468981a704. Latest consensus state: current_term: 1 leader_uuid: "5223da4d29f14eefae63ff468981a704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5223da4d29f14eefae63ff468981a704" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45309 } } }
I20260619 20:48:15.050647 32262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:15.050732 32254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:15.052379 32262 catalog_manager.cc:1347] Generated new cluster ID: f7e9908cf56b47a98a42c0347e7fa4d3
I20260619 20:48:15.052433 32262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:15.060398 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.050817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.13.195.126:45309)
0619 20:48:15.050978 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.050988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.051078 (+ 90us) server_negotiation.cc:407] Connection header received
0619 20:48:15.051206 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.051213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.051266 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.051382 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:15.052144 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.053150 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.054160 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.054616 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.054746 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.057106 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:15.057110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:15.057111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:15.057122 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:15.058848 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.059230 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.059233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.059234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.059334 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.059547 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.059548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.059548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.059784 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:15.059945 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.060051 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.060192 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":80,"threads_started":1}
I20260619 20:48:15.067945 32262 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:15.068632 32262 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:15.072989 32262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704: Generated new TSK 0
I20260619 20:48:15.073148 32262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:48:15.102820 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45309
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:15.185367 32275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:15.185604 32275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:15.185647 32275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:15.187305 32275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:15.187371 32275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:15.187415 32275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:48:15.188851 32275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:15.189131 32275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:15.189366 32275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:15.194315 32283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.194268 32280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.194346 32281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:15.194533 32275 server_base.cc:1034] running on GCE node
I20260619 20:48:15.194722 32275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:15.194916 32275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:15.196061 32275 hybrid_clock.cc:648] HybridClock initialized: now 1781902095196040 us; error 26 us; skew 500 ppm
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902095, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:15.198568 32275 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:48:15.199018 32275 webserver.cc:460] Webserver started at http://127.13.195.65:43009/ using document root <none> and password file <none>
I20260619 20:48:15.199203 32275 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:15.199242 32275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:15.199331 32275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:15.200310 32275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "3de8017192c5479f87f847695a467a73"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "2a94bc6a27192b2a35aa1788544d3bfa"
server_key_iv: "524ce15c61888a21223b76d40ca4cfe9"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.200604 32275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "3de8017192c5479f87f847695a467a73"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "2a94bc6a27192b2a35aa1788544d3bfa"
server_key_iv: "524ce15c61888a21223b76d40ca4cfe9"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.201861 32275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:15.202749 32290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.202924 32275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:48:15.203006 32275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "3de8017192c5479f87f847695a467a73"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "2a94bc6a27192b2a35aa1788544d3bfa"
server_key_iv: "524ce15c61888a21223b76d40ca4cfe9"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.203061 32275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:15.215726 32275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:15.217476 32275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:15.217588 32275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:15.217769 32275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:15.218014 32275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:15.218043 32275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.218065 32275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:15.218077 32275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.223680 32275 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:38099
I20260619 20:48:15.223764 32403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:38099 every 8 connection(s)
I20260619 20:48:15.224066 32275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:48:15.227779 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 32275
I20260619 20:48:15.227913 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:48:15.228137 14093 external_mini_cluster.cc:1428] Setting key 00be96400d3301001f803da27e6711d0
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902095, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:15.233858 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.224802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:58687 (local address 127.13.195.126:45309)
0619 20:48:15.224871 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.224873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.225040 (+ 167us) server_negotiation.cc:407] Connection header received
0619 20:48:15.225566 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.225567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.225575 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.225650 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:15.226634 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.227134 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.228255 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.228566 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.228628 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.231001 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:15.231003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:15.231003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:15.231011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:15.232366 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.232890 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.232891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.232891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.232938 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.233221 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.233222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.233222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.233321 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:15.233366 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.233727 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.233794 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:48:15.234196 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.224756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45309 (local address 127.13.195.65:58687)
0619 20:48:15.224914 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.224955 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.225479 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.225705 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.225708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.226268 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:15.226530 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.226537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.227232 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.227233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.228127 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.228129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.228655 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.228656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.228796 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.229357 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:15.229365 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:15.230903 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:15.232452 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:15.232460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:15.232462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:15.232812 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:15.233003 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:15.233004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:15.233004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:15.233149 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:15.233383 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:15.233390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:15.233574 (+ 184us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.233715 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.233907 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260619 20:48:15.234843 32404 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45309
I20260619 20:48:15.234951 32404 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:15.235157 32404 heartbeater.cc:511] Master 127.13.195.126:45309 requested a full tablet report, sending...
I20260619 20:48:15.235675 32213 ts_manager.cc:194] Registered new tserver with Master: 3de8017192c5479f87f847695a467a73 (127.13.195.65:38099)
I20260619 20:48:15.236312 32213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:58687
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:48:15.253993 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45309
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:15.335417 32409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:15.335626 32409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:15.335650 32409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:15.336794 32409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:15.336835 32409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:15.336861 32409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:48:15.338271 32409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:15.338524 32409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:15.338754 32409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:15.343636 32414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.343643 32417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.343633 32415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:15.343659 32409 server_base.cc:1034] running on GCE node
I20260619 20:48:15.344049 32409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:15.344290 32409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:15.345446 32409 hybrid_clock.cc:648] HybridClock initialized: now 1781902095345435 us; error 30 us; skew 500 ppm
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902095, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:15.347929 32409 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:48:15.348412 32409 webserver.cc:460] Webserver started at http://127.13.195.66:37705/ using document root <none> and password file <none>
I20260619 20:48:15.348635 32409 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:15.348703 32409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:15.348857 32409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:15.350082 32409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "88849c70a5f356f93ca4f484d7d4bbe2"
server_key_iv: "b5ad8da7abd7ba3fe00d54ce1a373c62"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.350565 32409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "88849c70a5f356f93ca4f484d7d4bbe2"
server_key_iv: "b5ad8da7abd7ba3fe00d54ce1a373c62"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.351846 32409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:15.352550 32424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.352763 32409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:15.352847 32409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "88849c70a5f356f93ca4f484d7d4bbe2"
server_key_iv: "b5ad8da7abd7ba3fe00d54ce1a373c62"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.352927 32409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:15.372444 32409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:15.374356 32409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:15.374555 32409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:15.374804 32409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:15.375129 32409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:15.375185 32409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.375224 32409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:15.375254 32409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.380878 32409 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:34665
I20260619 20:48:15.380959 32537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:34665 every 8 connection(s)
I20260619 20:48:15.381322 32409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902095, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:15.388542 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 32409
I20260619 20:48:15.388664 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:48:15.388895 14093 external_mini_cluster.cc:1428] Setting key a2aeb65a8fd97cd3168edeaefdfe91c8
I20260619 20:48:15.390717 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.382078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:60577 (local address 127.13.195.126:45309)
0619 20:48:15.382150 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.382152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.382380 (+ 228us) server_negotiation.cc:407] Connection header received
0619 20:48:15.382908 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.382909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.382917 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.382978 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:15.383965 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.384464 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.385426 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.385634 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.385689 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.387791 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:15.387792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:15.387792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:15.387799 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:15.389188 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.389649 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.389650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.389650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.389716 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.389953 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.389954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.389955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.390096 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:15.390170 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.390402 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.390553 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:48:15.390745 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.382065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45309 (local address 127.13.195.66:60577)
0619 20:48:15.382246 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.382289 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.382820 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.383033 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.383036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.383695 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:15.383888 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.383892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.384551 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.384551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.385352 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.385354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.385707 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.385708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.385804 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.386354 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:15.386363 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:15.387706 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:15.389270 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:15.389277 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:15.389279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:15.389567 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:15.389787 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:15.389787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:15.389788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:15.389888 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:15.390163 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:15.390168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:15.390280 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.390360 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.390547 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260619 20:48:15.391548 32538 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45309
I20260619 20:48:15.391667 32538 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:15.391867 32538 heartbeater.cc:511] Master 127.13.195.126:45309 requested a full tablet report, sending...
I20260619 20:48:15.392232 32213 ts_manager.cc:194] Registered new tserver with Master: 1a1bbadf733e4b9f9bcf3dca8655eb51 (127.13.195.66:34665)
I20260619 20:48:15.392637 32213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:60577
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:48:15.413489 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45309
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:42085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:15.496668 32543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:15.496922 32543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:15.496958 32543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:15.498144 32543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:15.498214 32543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:15.498257 32543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:48:15.499742 32543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:42085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:15.500013 32543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:15.500247 32543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:48:15.505640 32543 server_base.cc:1034] running on GCE node
W20260619 20:48:15.505632 32548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.505591 32551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.505601 32549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:15.506127 32543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:15.506374 32543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:15.507513 32543 hybrid_clock.cc:648] HybridClock initialized: now 1781902095507502 us; error 30 us; skew 500 ppm
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902095, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:15.510183 32543 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:48:15.510681 32543 webserver.cc:460] Webserver started at http://127.13.195.67:33813/ using document root <none> and password file <none>
I20260619 20:48:15.510921 32543 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:15.510967 32543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:15.511118 32543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:15.512185 32543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "4d351a21d7de444ba50e1b8f729ed6bd"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "2c3b21860ebedc237b02ac16eaa3929e"
server_key_iv: "cae625e4abd8254d3b318e02e3760d97"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.512539 32543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "4d351a21d7de444ba50e1b8f729ed6bd"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "2c3b21860ebedc237b02ac16eaa3929e"
server_key_iv: "cae625e4abd8254d3b318e02e3760d97"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.513863 32543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:15.514642 32558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.514816 32543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:15.514922 32543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "4d351a21d7de444ba50e1b8f729ed6bd"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "2c3b21860ebedc237b02ac16eaa3929e"
server_key_iv: "cae625e4abd8254d3b318e02e3760d97"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.514997 32543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:15.522562 32543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:15.524793 32543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:15.524956 32543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:15.525200 32543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:15.525514 32543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:15.525569 32543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.525609 32543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:15.525638 32543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.531665 32543 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:42427
I20260619 20:48:15.531718 32671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:42427 every 8 connection(s)
I20260619 20:48:15.532021 32543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:48:15 dist-test-slave-n2td krb5kdc[32168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902095, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:15.538223 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 32543
I20260619 20:48:15.538363 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:48:15.538583 14093 external_mini_cluster.cc:1428] Setting key 06110bac2494f6095128863cc089b8b4
I20260619 20:48:15.541208 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.532688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:41103 (local address 127.13.195.126:45309)
0619 20:48:15.532733 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.532735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.532953 (+ 218us) server_negotiation.cc:407] Connection header received
0619 20:48:15.533409 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.533409 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.533418 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.533477 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:15.534382 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.535003 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.536025 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.536216 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.536268 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.538353 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:15.538355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:15.538355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:15.538362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:15.540018 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.540457 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.540458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.540458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.540508 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.540718 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.540718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.540719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.540814 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:15.540862 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.541079 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.541145 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:15.541366 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.532660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45309 (local address 127.13.195.67:41103)
0619 20:48:15.532825 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.532866 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.533330 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.533550 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.533559 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.534100 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:15.534303 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.534307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.535104 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.535105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.535937 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.535940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.536291 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.536291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.536391 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.536850 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:15.536868 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:15.538250 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:15.540093 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:15.540100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:15.540102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:15.540392 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:15.540567 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:15.540567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:15.540568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:15.540663 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:15.540866 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:15.540869 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:15.540974 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.541056 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.541169 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260619 20:48:15.541810 32672 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45309
I20260619 20:48:15.541908 32672 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:15.542081 32672 heartbeater.cc:511] Master 127.13.195.126:45309 requested a full tablet report, sending...
I20260619 20:48:15.542482 32213 ts_manager.cc:194] Registered new tserver with Master: 4d351a21d7de444ba50e1b8f729ed6bd (127.13.195.67:42427)
I20260619 20:48:15.542845 32213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:41103
I20260619 20:48:15.550578 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:48:15.558035 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.551963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38910 (local address 127.13.195.126:45309)
0619 20:48:15.552003 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.552004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.552125 (+ 121us) server_negotiation.cc:407] Connection header received
0619 20:48:15.552172 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.552173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.552181 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.552231 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:15.552901 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.553360 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.554066 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.554242 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.554294 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.555559 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:15.555561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:15.555561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:15.555568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:15.556816 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.557204 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.557205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.557205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.557261 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:15.557485 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:15.557486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:15.557486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:15.557623 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:15.557738 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.557870 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.557933 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:48:15.559809 32213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:48:15.561556 32213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:48:15.567935 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.563876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:42427 (local address 127.0.0.1:54958)
0619 20:48:15.564179 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.564189 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.564248 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.564508 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.564509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.564511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:15.564568 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.564570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.565888 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.565889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.566852 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.566855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.567602 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.567603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.567689 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.567703 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.567803 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.567860 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":28,"thread_start_us":50,"threads_started":1}
I20260619 20:48:15.567950 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.563982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54958 (local address 127.13.195.67:42427)
0619 20:48:15.564152 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.564158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.564234 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:48:15.564324 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.564326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.564342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.564428 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:15.564656 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.565767 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.566944 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.567497 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.567597 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.567657 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.567820 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.567862 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260619 20:48:15.568164 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.563648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38099 (local address 127.0.0.1:37174)
0619 20:48:15.563780 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.563819 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.563877 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.564389 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.564392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.564394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:15.564463 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.564465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.565589 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.565590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.566865 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.566868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.567878 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.567879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.567953 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.567956 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.567986 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.568024 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260619 20:48:15.568238 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.563655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:34665 (local address 127.0.0.1:46092)
0619 20:48:15.563980 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.563989 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.564025 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.564587 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.564588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.564590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:15.564622 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.564624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.565903 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.565903 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.567028 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.567030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.567888 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.567888 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.567944 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.567949 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.567980 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.568022 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":42,"thread_start_us":80,"threads_started":1}
I20260619 20:48:15.568318 32684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.564017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.13.195.65:38099)
0619 20:48:15.564187 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.564192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.564203 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:48:15.564244 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.564246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.564263 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.564543 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:15.564632 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.565306 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.567015 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.567769 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.568088 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.568155 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.568197 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.568241 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260619 20:48:15.568338 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.564240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.13.195.66:34665)
0619 20:48:15.564427 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.564431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.564440 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:48:15.564477 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.564479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.564494 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.564691 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:15.564759 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.565797 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.567141 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.567804 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.568129 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.568200 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.568224 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.568263 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260619 20:48:15.569082 32472 tablet_service.cc:1467] Processing CreateTablet for tablet 95978afcc585434484cddff3c4f6f04e (DEFAULT_TABLE table=test-table [id=02f29646fe2549d8b6a91470f1acdecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:15.569228 32606 tablet_service.cc:1467] Processing CreateTablet for tablet 95978afcc585434484cddff3c4f6f04e (DEFAULT_TABLE table=test-table [id=02f29646fe2549d8b6a91470f1acdecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:15.569229 32338 tablet_service.cc:1467] Processing CreateTablet for tablet 95978afcc585434484cddff3c4f6f04e (DEFAULT_TABLE table=test-table [id=02f29646fe2549d8b6a91470f1acdecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:15.569494 32472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95978afcc585434484cddff3c4f6f04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:15.569559 32338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95978afcc585434484cddff3c4f6f04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:15.569566 32606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95978afcc585434484cddff3c4f6f04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:15.572057 32686 tablet_bootstrap.cc:492] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Bootstrap starting.
I20260619 20:48:15.572806 32687 tablet_bootstrap.cc:492] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Bootstrap starting.
I20260619 20:48:15.572983 32686 tablet_bootstrap.cc:654] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:15.573297 32688 tablet_bootstrap.cc:492] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73: Bootstrap starting.
I20260619 20:48:15.573590 32686 log.cc:824] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:15.573796 32687 tablet_bootstrap.cc:654] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:15.573962 32688 tablet_bootstrap.cc:654] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:15.574358 32687 log.cc:824] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:15.574530 32686 tablet_bootstrap.cc:492] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: No bootstrap required, opened a new log
I20260619 20:48:15.574538 32688 log.cc:824] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:15.574632 32686 ts_tablet_manager.cc:1397] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:48:15.575245 32688 tablet_bootstrap.cc:492] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73: No bootstrap required, opened a new log
I20260619 20:48:15.575323 32688 ts_tablet_manager.cc:1397] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:48:15.576263 32686 raft_consensus.cc:348] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.576399 32686 raft_consensus.cc:374] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:15.576421 32686 raft_consensus.cc:729] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1bbadf733e4b9f9bcf3dca8655eb51, State: Initialized, Role: FOLLOWER
I20260619 20:48:15.576547 32686 consensus_queue.cc:260] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.576846 32686 ts_tablet_manager.cc:1428] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:48:15.576875 32538 heartbeater.cc:503] Master 127.13.195.126:45309 was elected leader, sending a full tablet report...
I20260619 20:48:15.577227 32688 raft_consensus.cc:348] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.577409 32687 tablet_bootstrap.cc:492] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: No bootstrap required, opened a new log
I20260619 20:48:15.577410 32688 raft_consensus.cc:374] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:15.577452 32687 ts_tablet_manager.cc:1397] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 20:48:15.577448 32688 raft_consensus.cc:729] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de8017192c5479f87f847695a467a73, State: Initialized, Role: FOLLOWER
I20260619 20:48:15.577553 32688 consensus_queue.cc:260] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.577850 32688 ts_tablet_manager.cc:1428] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:48:15.577909 32404 heartbeater.cc:503] Master 127.13.195.126:45309 was elected leader, sending a full tablet report...
I20260619 20:48:15.579275 32687 raft_consensus.cc:348] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.579460 32687 raft_consensus.cc:374] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:15.579495 32687 raft_consensus.cc:729] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d351a21d7de444ba50e1b8f729ed6bd, State: Initialized, Role: FOLLOWER
I20260619 20:48:15.579584 32687 consensus_queue.cc:260] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.579789 32687 ts_tablet_manager.cc:1428] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:48:15.579846 32672 heartbeater.cc:503] Master 127.13.195.126:45309 was elected leader, sending a full tablet report...
W20260619 20:48:15.631906 32539 tablet.cc:2369] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:48:15.681303 32694 raft_consensus.cc:482] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:15.681445 32694 raft_consensus.cc:504] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.681880 32694 leader_election.cc:288] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3de8017192c5479f87f847695a467a73 (127.13.195.65:38099), 1a1bbadf733e4b9f9bcf3dca8655eb51 (127.13.195.66:34665)
I20260619 20:48:15.685262 32684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.682112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:47505 (local address 127.13.195.65:38099)
0619 20:48:15.682163 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.682165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.682177 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:48:15.682329 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.682330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.682347 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.682431 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:15.682717 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.683457 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.684372 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.684940 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.684995 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.685003 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.685179 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.685209 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:48:15.685273 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.682052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38099 (local address 127.13.195.67:47505)
0619 20:48:15.682112 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.682125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.682177 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.682475 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.682477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.682479 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:15.682576 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.682578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.683537 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.683538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.684284 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.684285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.685017 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.685018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.685098 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.685102 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.685163 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.685200 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:48:15.685725 32695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.682101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:34665 (local address 127.13.195.67:54755)
0619 20:48:15.682355 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.682366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:15.682503 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:15.682775 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:15.682776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:15.682778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:15.682858 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.682860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.683778 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.683779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.684762 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:15.684765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.685497 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.685498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:15.685573 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.685578 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:48:15.685644 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:48:15.685673 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":126,"threads_started":1}
I20260619 20:48:15.685686 32358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95978afcc585434484cddff3c4f6f04e" candidate_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de8017192c5479f87f847695a467a73" is_pre_election: true
I20260619 20:48:15.685771 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.682248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:54755 (local address 127.13.195.66:34665)
0619 20:48:15.682296 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.682298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.682417 (+ 119us) server_negotiation.cc:407] Connection header received
0619 20:48:15.682608 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.682609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.682619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.682709 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:15.682966 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.683696 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.684850 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.685396 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.685517 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.685527 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.685670 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.685712 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:48:15.685844 32358 raft_consensus.cc:2454] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d351a21d7de444ba50e1b8f729ed6bd in term 0.
I20260619 20:48:15.686093 32561 leader_election.cc:302] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3de8017192c5479f87f847695a467a73, 4d351a21d7de444ba50e1b8f729ed6bd; no voters:
I20260619 20:48:15.686110 32492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95978afcc585434484cddff3c4f6f04e" candidate_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" is_pre_election: true
I20260619 20:48:15.686239 32492 raft_consensus.cc:2454] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d351a21d7de444ba50e1b8f729ed6bd in term 0.
I20260619 20:48:15.686266 32694 raft_consensus.cc:2781] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:48:15.686342 32694 raft_consensus.cc:482] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:48:15.686388 32694 raft_consensus.cc:3037] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:15.687127 32694 raft_consensus.cc:504] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.687270 32694 leader_election.cc:288] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [CANDIDATE]: Term 1 election: Requested vote from peers 3de8017192c5479f87f847695a467a73 (127.13.195.65:38099), 1a1bbadf733e4b9f9bcf3dca8655eb51 (127.13.195.66:34665)
I20260619 20:48:15.687458 32492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95978afcc585434484cddff3c4f6f04e" candidate_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51"
I20260619 20:48:15.687465 32358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95978afcc585434484cddff3c4f6f04e" candidate_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de8017192c5479f87f847695a467a73"
I20260619 20:48:15.687552 32492 raft_consensus.cc:3037] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:15.687551 32358 raft_consensus.cc:3037] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:15.688265 32358 raft_consensus.cc:2454] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d351a21d7de444ba50e1b8f729ed6bd in term 1.
I20260619 20:48:15.688354 32492 raft_consensus.cc:2454] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d351a21d7de444ba50e1b8f729ed6bd in term 1.
I20260619 20:48:15.688444 32561 leader_election.cc:302] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3de8017192c5479f87f847695a467a73, 4d351a21d7de444ba50e1b8f729ed6bd; no voters:
I20260619 20:48:15.688539 32694 raft_consensus.cc:2781] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:15.688690 32694 raft_consensus.cc:686] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 1 LEADER]: Becoming Leader. State: Replica: 4d351a21d7de444ba50e1b8f729ed6bd, State: Running, Role: LEADER
I20260619 20:48:15.688797 32694 consensus_queue.cc:237] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } }
I20260619 20:48:15.689622 32213 catalog_manager.cc:5495] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d351a21d7de444ba50e1b8f729ed6bd (127.13.195.67). New cstate: current_term: 1 leader_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d351a21d7de444ba50e1b8f729ed6bd" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 42427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 } health_report { overall_health: UNKNOWN } } }
I20260619 20:48:15.716542 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.714027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.13.195.67:42427)
0619 20:48:15.714099 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.714102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.714126 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:48:15.714188 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.714191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.714215 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.714302 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:15.714471 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.714939 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.715721 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.715910 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.715987 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.716169 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:15.716274 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:15.716391 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.716422 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.716434 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:48:15.718448 32358 raft_consensus.cc:1261] T 95978afcc585434484cddff3c4f6f04e P 3de8017192c5479f87f847695a467a73 [term 1 FOLLOWER]: Refusing update from remote peer 4d351a21d7de444ba50e1b8f729ed6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:15.718473 32492 raft_consensus.cc:1261] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 1 FOLLOWER]: Refusing update from remote peer 4d351a21d7de444ba50e1b8f729ed6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:15.718864 32696 consensus_queue.cc:1035] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3de8017192c5479f87f847695a467a73" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:15.718999 32694 consensus_queue.cc:1035] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a1bbadf733e4b9f9bcf3dca8655eb51" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 34665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:15.721347 32699 mvcc.cc:205] Tried to move back new op lower bound from 7298670984060006400 to 7298670983941922816. Current Snapshot: MvccSnapshot[applied={T|T < 7298670984060006400}]
I20260619 20:48:15.721712 32701 mvcc.cc:205] Tried to move back new op lower bound from 7298670984060006400 to 7298670983941922816. Current Snapshot: MvccSnapshot[applied={T|T < 7298670984060006400}]
I20260619 20:48:15.722358 32700 mvcc.cc:205] Tried to move back new op lower bound from 7298670984060006400 to 7298670983941922816. Current Snapshot: MvccSnapshot[applied={T|T < 7298670984060006400}]
I20260619 20:48:15.724365 32213 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:15.724457 32213 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:15.725294 32213 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5223da4d29f14eefae63ff468981a704: Sending DeleteTablet for 3 replicas of tablet 95978afcc585434484cddff3c4f6f04e
I20260619 20:48:15.725663 32472 tablet_service.cc:1514] Processing DeleteTablet for tablet 95978afcc585434484cddff3c4f6f04e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:15 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:46092
I20260619 20:48:15.725760 32338 tablet_service.cc:1514] Processing DeleteTablet for tablet 95978afcc585434484cddff3c4f6f04e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:15 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:37174
I20260619 20:48:15.725939 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 32275
I20260619 20:48:15.725967 32606 tablet_service.cc:1514] Processing DeleteTablet for tablet 95978afcc585434484cddff3c4f6f04e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:15 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:54958
I20260619 20:48:15.726601 32710 tablet_replica.cc:330] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: stopping tablet replica
I20260619 20:48:15.726720 32710 raft_consensus.cc:2229] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:48:15.726850 32710 raft_consensus.cc:2258] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:15.727303 32710 ts_tablet_manager.cc:1905] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:15.729055 32710 ts_tablet_manager.cc:1918] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:15.729133 32710 log.cc:1196] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/95978afcc585434484cddff3c4f6f04e
I20260619 20:48:15.729403 32710 ts_tablet_manager.cc:1939] T 95978afcc585434484cddff3c4f6f04e P 4d351a21d7de444ba50e1b8f729ed6bd: Deleting consensus metadata
I20260619 20:48:15.729846 32199 catalog_manager.cc:4841] TS 4d351a21d7de444ba50e1b8f729ed6bd (127.13.195.67:42427): tablet 95978afcc585434484cddff3c4f6f04e (table test-table [id=02f29646fe2549d8b6a91470f1acdecf]) successfully deleted
I20260619 20:48:15.730578 32708 tablet_replica.cc:330] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: stopping tablet replica
I20260619 20:48:15.730701 32708 raft_consensus.cc:2229] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:15.730813 32708 raft_consensus.cc:2258] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:15.731240 32708 ts_tablet_manager.cc:1905] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:15.732820 32708 ts_tablet_manager.cc:1918] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:15.732893 32708 log.cc:1196] T 95978afcc585434484cddff3c4f6f04e P 1a1bbadf733e4b9f9bcf3dca8655eb51: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/95978afcc585434484cddff3c4f6f04e
W20260619 20:48:15.733132 32199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:38099 (error 108)
I20260619 20:48:15.733327 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 32409
I20260619 20:48:15.734020 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.733467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38099 (local address 127.0.0.1:37176)
0619 20:48:15.733937 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.733977 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447}
W20260619 20:48:15.735562 32199 catalog_manager.cc:4584] TS 3de8017192c5479f87f847695a467a73 (127.13.195.65:38099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95978afcc585434484cddff3c4f6f04e: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38099: connect: Connection refused (error 111)
I20260619 20:48:15.738896 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 32543
I20260619 20:48:15.742615 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.738821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:34665 (local address 127.0.0.1:46102)
0619 20:48:15.742531 (+ 3710us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:15.742579 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.66:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3678}
I20260619 20:48:15.743643 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 32181
2026-06-19T20:48:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1979 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T20:48:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:15Z Disabled control of system clock
I20260619 20:48:15.771039 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45123
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:45123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:15.852052 32715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:15.852262 32715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:15.852283 32715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:15.853327 32715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:15.853363 32715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:15.853375 32715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:15.853389 32715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:15.853400 32715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:15.854671 32715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:45123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:15.854916 32715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:15.855154 32715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:15.859807 32720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:15.859874 32715 server_base.cc:1034] running on GCE node
W20260619 20:48:15.859889 32723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:15.859861 32721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:15.860499 32715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:15.860729 32715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:15.861903 32715 hybrid_clock.cc:648] HybridClock initialized: now 1781902095861892 us; error 51 us; skew 500 ppm
I20260619 20:48:15.863351 32715 webserver.cc:460] Webserver started at http://127.13.195.126:34651/ using document root <none> and password file <none>
I20260619 20:48:15.863596 32715 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:15.863664 32715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:15.863801 32715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:15.864849 32715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "dedb8c31178741ecacea423a6e6d491f"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "679bcf8502a8fee4e8b5752fac093694"
server_key_iv: "47abe7ec1c4f3229c0146bd7acf82727"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.865252 32715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "dedb8c31178741ecacea423a6e6d491f"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "679bcf8502a8fee4e8b5752fac093694"
server_key_iv: "47abe7ec1c4f3229c0146bd7acf82727"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.866602 32715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:15.867333 32729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:15.867533 32715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:15.867631 32715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "dedb8c31178741ecacea423a6e6d491f"
format_stamp: "Formatted at 2026-06-19 20:48:15 on dist-test-slave-n2td"
server_key: "679bcf8502a8fee4e8b5752fac093694"
server_key_iv: "47abe7ec1c4f3229c0146bd7acf82727"
server_key_version: "encryptionkey@0"
I20260619 20:48:15.867717 32715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:15.884027 32715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:15.884379 32715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:15.884541 32715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:15.888573 32715 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:45123
I20260619 20:48:15.888612 313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:45123 every 8 connection(s)
I20260619 20:48:15.888988 32715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:15.889927 314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:15.892272 314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f: Bootstrap starting.
I20260619 20:48:15.892863 314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:15.893163 314 log.cc:824] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:15.893874 314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f: No bootstrap required, opened a new log
I20260619 20:48:15.895474 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 32715
I20260619 20:48:15.895571 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:15.895509 314 raft_consensus.cc:348] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dedb8c31178741ecacea423a6e6d491f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45123 } }
I20260619 20:48:15.895728 314 raft_consensus.cc:374] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:15.895793 14093 external_mini_cluster.cc:1428] Setting key 4db1e5af2882d4cec29f5f0586231cbe
I20260619 20:48:15.895797 314 raft_consensus.cc:729] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dedb8c31178741ecacea423a6e6d491f, State: Initialized, Role: FOLLOWER
I20260619 20:48:15.895946 314 consensus_queue.cc:260] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dedb8c31178741ecacea423a6e6d491f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45123 } }
I20260619 20:48:15.896052 314 raft_consensus.cc:388] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:15.896090 314 raft_consensus.cc:482] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:15.896143 314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:15.897048 314 raft_consensus.cc:504] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dedb8c31178741ecacea423a6e6d491f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45123 } }
I20260619 20:48:15.897152 314 leader_election.cc:302] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dedb8c31178741ecacea423a6e6d491f; no voters:
I20260619 20:48:15.897315 314 leader_election.cc:288] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:15.897375 319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:15.897555 319 raft_consensus.cc:686] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [term 1 LEADER]: Becoming Leader. State: Replica: dedb8c31178741ecacea423a6e6d491f, State: Running, Role: LEADER
I20260619 20:48:15.897581 314 sys_catalog.cc:564] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:15.897650 319 consensus_queue.cc:237] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dedb8c31178741ecacea423a6e6d491f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45123 } }
I20260619 20:48:15.897977 320 sys_catalog.cc:455] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dedb8c31178741ecacea423a6e6d491f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dedb8c31178741ecacea423a6e6d491f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45123 } } }
I20260619 20:48:15.898057 320 sys_catalog.cc:458] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:15.898218 321 sys_catalog.cc:455] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dedb8c31178741ecacea423a6e6d491f. Latest consensus state: current_term: 1 leader_uuid: "dedb8c31178741ecacea423a6e6d491f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dedb8c31178741ecacea423a6e6d491f" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 45123 } } }
I20260619 20:48:15.898283 321 sys_catalog.cc:458] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:15.898386 327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:15.898864 327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:15.900499 327 catalog_manager.cc:1347] Generated new cluster ID: e230f862fff54cdaba4af2b8bb3f0984
I20260619 20:48:15.900550 327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:15.902272 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:15.896310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.13.195.126:45123)
0619 20:48:15.896495 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:15.896505 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:15.896538 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:48:15.896656 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:15.896663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:15.896742 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:15.896886 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:15.897058 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.898882 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.900047 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:15.900642 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:15.900762 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:15.901473 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:15.901477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:15.901477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:15.901682 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:15.901723 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:15.901739 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:15.901810 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:15.901878 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 20:48:15.902053 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260619 20:48:15.914780 327 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:15.915514 327 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:15.920095 327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f: Generated new TSK 0
I20260619 20:48:15.920320 327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:15.926880 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45123
--builtin_ntp_servers=127.13.195.84:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:16.007562 338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:16.007794 338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:16.007829 338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:16.008981 338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:16.009049 338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:16.009102 338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:48:16.010468 338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:16.010720 338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:16.011013 338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:16.015787 346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:16.015776 344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:16.015988 338 server_base.cc:1034] running on GCE node
W20260619 20:48:16.015779 343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:16.016260 338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:16.016494 338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:16.017654 338 hybrid_clock.cc:648] HybridClock initialized: now 1781902096017641 us; error 29 us; skew 500 ppm
I20260619 20:48:16.019052 338 webserver.cc:460] Webserver started at http://127.13.195.65:41727/ using document root <none> and password file <none>
I20260619 20:48:16.019279 338 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:16.019348 338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:16.019484 338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:16.020540 338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "7173eb6f2b34427a9adaac0aca74a329"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "c5c8649633f560af2098348d73d174fb"
server_key_iv: "d7d3006ff4ca840b8f285e71bc57cdf1"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.021072 338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "7173eb6f2b34427a9adaac0aca74a329"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "c5c8649633f560af2098348d73d174fb"
server_key_iv: "d7d3006ff4ca840b8f285e71bc57cdf1"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.022581 338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260619 20:48:16.023419 352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.023626 338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:16.023723 338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "7173eb6f2b34427a9adaac0aca74a329"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "c5c8649633f560af2098348d73d174fb"
server_key_iv: "d7d3006ff4ca840b8f285e71bc57cdf1"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.023813 338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:16.044378 338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:16.044719 338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:16.044875 338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:16.045136 338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:16.045462 338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:16.045632 338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.045724 338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:16.045758 338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.051709 338 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:43539
I20260619 20:48:16.051749 466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:43539 every 8 connection(s)
I20260619 20:48:16.052099 338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:48:16.056939 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.052818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:44829 (local address 127.13.195.126:45123)
0619 20:48:16.052861 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.052864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.053127 (+ 263us) server_negotiation.cc:407] Connection header received
0619 20:48:16.053584 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.053585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.053597 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.053652 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:16.054270 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.054824 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.055753 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.055934 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.055984 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.056733 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:16.056734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:16.056735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:16.056740 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:16.056753 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:16.056756 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:16.056799 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.056858 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.056883 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:16.057166 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.052822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45123 (local address 127.13.195.65:44829)
0619 20:48:16.053000 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.053033 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.053503 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.053703 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.053706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.053713 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:16.054204 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.054208 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.054904 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.054905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.055685 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.055687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.056002 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.056003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.056109 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.056610 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:16.056610 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:16.056637 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:16.056657 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:16.056798 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:16.056800 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:16.056802 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.056847 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.056918 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260619 20:48:16.057737 467 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45123
I20260619 20:48:16.057813 467 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:16.057997 467 heartbeater.cc:511] Master 127.13.195.126:45123 requested a full tablet report, sending...
I20260619 20:48:16.058530 32746 ts_manager.cc:194] Registered new tserver with Master: 7173eb6f2b34427a9adaac0aca74a329 (127.13.195.65:43539)
I20260619 20:48:16.059135 32746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.65:44829
I20260619 20:48:16.061972 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 338
I20260619 20:48:16.062064 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:48:16.062211 14093 external_mini_cluster.cc:1428] Setting key efe24ebc19df4a850ab21ea759fb5ed1
I20260619 20:48:16.063153 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45123
--builtin_ntp_servers=127.13.195.84:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:16.142894 470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:16.143126 470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:16.143147 470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:16.144311 470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:16.144352 470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:16.144376 470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:48:16.145725 470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:16.145980 470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:16.146307 470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:16.151129 478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:16.151108 475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:16.151134 476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:16.151443 470 server_base.cc:1034] running on GCE node
I20260619 20:48:16.151681 470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:16.151914 470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:16.153071 470 hybrid_clock.cc:648] HybridClock initialized: now 1781902096153045 us; error 37 us; skew 500 ppm
I20260619 20:48:16.154479 470 webserver.cc:460] Webserver started at http://127.13.195.66:42799/ using document root <none> and password file <none>
I20260619 20:48:16.154742 470 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:16.154812 470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:16.154954 470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:16.155992 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "180694e35ee84817a662d8b6a7e5e97b"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "347061b90e78b9c9656b84dfae9357e6"
server_key_iv: "1c1e2785c65c20a9ae6ad27ae295d02b"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.156370 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "180694e35ee84817a662d8b6a7e5e97b"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "347061b90e78b9c9656b84dfae9357e6"
server_key_iv: "1c1e2785c65c20a9ae6ad27ae295d02b"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.157703 470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:16.158483 484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.158669 470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:16.158766 470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "180694e35ee84817a662d8b6a7e5e97b"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "347061b90e78b9c9656b84dfae9357e6"
server_key_iv: "1c1e2785c65c20a9ae6ad27ae295d02b"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.158854 470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:16.180187 470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:16.180521 470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:16.180676 470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:16.180907 470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:16.181252 470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:16.181308 470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.181350 470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:16.181378 470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.187253 470 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:46451
I20260619 20:48:16.187305 598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:46451 every 8 connection(s)
I20260619 20:48:16.187647 470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:48:16.192947 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.188383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:40593 (local address 127.13.195.126:45123)
0619 20:48:16.188435 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.188438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.188671 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:48:16.189210 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.189212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.189225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.189278 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:16.189984 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.190510 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.191511 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.191725 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.191803 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.192643 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:16.192645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:16.192645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:16.192650 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:16.192665 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:16.192670 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:16.192741 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.192836 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.192867 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:48:16.193173 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.188348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45123 (local address 127.13.195.66:40593)
0619 20:48:16.188527 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.188576 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.189096 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.189354 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.189358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.189370 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:16.189881 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.189886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.190619 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.190621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.191413 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.191416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.191822 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.191823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.191960 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.192462 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:16.192463 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:16.192491 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:16.192514 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:16.192741 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:16.192748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:16.192751 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.192837 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.192941 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260619 20:48:16.193624 599 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45123
I20260619 20:48:16.193739 599 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:16.193924 599 heartbeater.cc:511] Master 127.13.195.126:45123 requested a full tablet report, sending...
I20260619 20:48:16.194293 32746 ts_manager.cc:194] Registered new tserver with Master: 180694e35ee84817a662d8b6a7e5e97b (127.13.195.66:46451)
I20260619 20:48:16.194692 32746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.66:40593
I20260619 20:48:16.197817 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 470
I20260619 20:48:16.197929 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:48:16.198148 14093 external_mini_cluster.cc:1428] Setting key 1e5a4b93245293e34f41aef584b97dcc
I20260619 20:48:16.199050 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45123
--builtin_ntp_servers=127.13.195.84:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:16.279450 602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:16.279668 602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:16.279690 602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:16.280747 602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:16.280781 602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:16.280805 602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:48:16.282074 602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:45123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:16.282256 602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:16.282574 602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:16.287333 608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:16.287341 607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:16.287341 610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:16.287742 602 server_base.cc:1034] running on GCE node
I20260619 20:48:16.287966 602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:16.288164 602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:16.289315 602 hybrid_clock.cc:648] HybridClock initialized: now 1781902096289306 us; error 27 us; skew 500 ppm
I20260619 20:48:16.290913 602 webserver.cc:460] Webserver started at http://127.13.195.67:46815/ using document root <none> and password file <none>
I20260619 20:48:16.291159 602 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:16.291206 602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:16.291350 602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:16.292392 602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "dd6842d031ca4699a5dd2dab1acd30e5"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "6742b2cdb571f9114a3d7966406df13d"
server_key_iv: "1d70fa9e437118bc28c9060413494fc8"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.292757 602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "dd6842d031ca4699a5dd2dab1acd30e5"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "6742b2cdb571f9114a3d7966406df13d"
server_key_iv: "1d70fa9e437118bc28c9060413494fc8"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.294049 602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:16.294838 616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.295003 602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.295105 602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "dd6842d031ca4699a5dd2dab1acd30e5"
format_stamp: "Formatted at 2026-06-19 20:48:16 on dist-test-slave-n2td"
server_key: "6742b2cdb571f9114a3d7966406df13d"
server_key_iv: "1d70fa9e437118bc28c9060413494fc8"
server_key_version: "encryptionkey@0"
I20260619 20:48:16.295183 602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:16.304323 602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:16.304611 602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:16.304756 602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:16.304965 602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:16.305282 602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:16.305343 602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.305384 602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:16.305413 602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:16.311164 602 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:36443
I20260619 20:48:16.311240 729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:36443 every 8 connection(s)
I20260619 20:48:16.311564 602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:48:16.313766 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 602
I20260619 20:48:16.313849 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:48:16.314087 14093 external_mini_cluster.cc:1428] Setting key 4d6898e79f5bd33b6017534c6a47db17
I20260619 20:48:16.317524 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.312327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:35373 (local address 127.13.195.126:45123)
0619 20:48:16.312374 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.312377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.312583 (+ 206us) server_negotiation.cc:407] Connection header received
0619 20:48:16.313116 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.313118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.313131 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.313184 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:16.313919 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.314652 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.315828 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.316130 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.316204 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.317221 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:16.317223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:16.317223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:16.317228 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:16.317244 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:16.317248 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:16.317306 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.317394 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.317425 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:16.317756 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.312290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:45123 (local address 127.13.195.67:35373)
0619 20:48:16.312457 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.312491 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.313000 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.313255 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.313260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.313271 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:16.313783 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.313788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.314799 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.314800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.315749 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.315754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.316226 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.316227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.316397 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.317037 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:16.317038 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:16.317078 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:16.317109 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:16.317314 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:16.317318 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:16.317321 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.317394 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.317501 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260619 20:48:16.318274 730 heartbeater.cc:347] Connected to a master server at 127.13.195.126:45123
I20260619 20:48:16.318380 730 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:16.318595 730 heartbeater.cc:511] Master 127.13.195.126:45123 requested a full tablet report, sending...
I20260619 20:48:16.318976 32746 ts_manager.cc:194] Registered new tserver with Master: dd6842d031ca4699a5dd2dab1acd30e5 (127.13.195.67:36443)
I20260619 20:48:16.319358 32746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.67:35373
I20260619 20:48:16.325881 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:48:16.330065 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.327383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.13.195.126:45123)
0619 20:48:16.327456 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.327458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.327512 (+ 54us) server_negotiation.cc:407] Connection header received
0619 20:48:16.327568 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.327569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.327579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.327625 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:16.327776 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.328238 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.328933 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.329104 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.329152 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.329746 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:16.329747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:16.329748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:16.329752 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:16.329764 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:16.329768 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:16.329851 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.329911 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.329920 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:48:16.331586 32746 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:48:16.333030 32746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:48:16.339418 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.335535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.13.195.66:46451)
0619 20:48:16.335693 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.335698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.335709 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:48:16.335749 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.335751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.335766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.335848 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:16.336083 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.337260 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.338498 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.338985 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.339075 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.339171 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.339289 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.339326 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260619 20:48:16.339443 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.335204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:46451 (local address 127.0.0.1:41148)
0619 20:48:16.335431 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.335440 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.335492 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.335915 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.335916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.335928 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:16.335985 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.335987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.337361 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.337362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.338409 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.338411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.339080 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.339082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.339175 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.339197 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.339271 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.339324 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":6,"thread_start_us":119,"threads_started":1}
I20260619 20:48:16.339584 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.335204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:36443 (local address 127.0.0.1:43624)
0619 20:48:16.335593 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.335600 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.335637 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.336004 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.336005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.336007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:16.336042 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.336044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.337383 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.337384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.338143 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.338146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.339104 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.339105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.339174 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.339196 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.339248 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.339499 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":207,"thread_start_us":117,"threads_started":1}
I20260619 20:48:16.339457 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.335197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:43539 (local address 127.0.0.1:38886)
0619 20:48:16.335385 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.335426 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.335492 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.335909 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.335911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.335914 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:16.335977 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.335978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.336913 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.336914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.338071 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.338074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.339103 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.339105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.339185 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.339193 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.339248 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.339322 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260619 20:48:16.339974 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.335275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.13.195.65:43539)
0619 20:48:16.335746 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.335750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.335759 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:48:16.335796 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.335798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.335812 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.336067 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:16.336131 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.336822 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.338179 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.338987 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.339696 (+ 709us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.339793 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.339822 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.339871 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":78,"threads_started":1}
I20260619 20:48:16.340167 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.335465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.13.195.67:36443)
0619 20:48:16.335722 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.335726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.335736 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:16.335769 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.335771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.335783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.336143 (+ 360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:16.336204 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.337260 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.338257 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.338988 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.339303 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.339396 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.339424 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.339461 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":78,"threads_started":1}
I20260619 20:48:16.340162 533 tablet_service.cc:1467] Processing CreateTablet for tablet 5357b0a3cda14380bfc82b14a453ed80 (DEFAULT_TABLE table=test-table [id=d8e5a633964e47ff937995467e0872cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:16.340565 533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5357b0a3cda14380bfc82b14a453ed80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:16.340885 664 tablet_service.cc:1467] Processing CreateTablet for tablet 5357b0a3cda14380bfc82b14a453ed80 (DEFAULT_TABLE table=test-table [id=d8e5a633964e47ff937995467e0872cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:16.341217 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5357b0a3cda14380bfc82b14a453ed80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:16.343101 400 tablet_service.cc:1467] Processing CreateTablet for tablet 5357b0a3cda14380bfc82b14a453ed80 (DEFAULT_TABLE table=test-table [id=d8e5a633964e47ff937995467e0872cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:16.343366 744 tablet_bootstrap.cc:492] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Bootstrap starting.
I20260619 20:48:16.343456 400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5357b0a3cda14380bfc82b14a453ed80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:16.343945 744 tablet_bootstrap.cc:654] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:16.344221 744 log.cc:824] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:16.344288 745 tablet_bootstrap.cc:492] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Bootstrap starting.
I20260619 20:48:16.344873 745 tablet_bootstrap.cc:654] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:16.345006 744 tablet_bootstrap.cc:492] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: No bootstrap required, opened a new log
I20260619 20:48:16.345077 744 ts_tablet_manager.cc:1397] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:48:16.345191 745 log.cc:824] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:16.346019 745 tablet_bootstrap.cc:492] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: No bootstrap required, opened a new log
I20260619 20:48:16.346091 745 ts_tablet_manager.cc:1397] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:48:16.346496 748 tablet_bootstrap.cc:492] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329: Bootstrap starting.
I20260619 20:48:16.347028 748 tablet_bootstrap.cc:654] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:16.347335 748 log.cc:824] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:16.348024 748 tablet_bootstrap.cc:492] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329: No bootstrap required, opened a new log
I20260619 20:48:16.348109 748 ts_tablet_manager.cc:1397] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:48:16.348306 744 raft_consensus.cc:348] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.348480 744 raft_consensus.cc:374] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:16.348517 744 raft_consensus.cc:729] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 180694e35ee84817a662d8b6a7e5e97b, State: Initialized, Role: FOLLOWER
I20260619 20:48:16.348656 744 consensus_queue.cc:260] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.348687 745 raft_consensus.cc:348] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.348840 745 raft_consensus.cc:374] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:16.348879 745 raft_consensus.cc:729] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6842d031ca4699a5dd2dab1acd30e5, State: Initialized, Role: FOLLOWER
I20260619 20:48:16.348922 744 ts_tablet_manager.cc:1428] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 20:48:16.349010 745 consensus_queue.cc:260] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.349046 599 heartbeater.cc:503] Master 127.13.195.126:45123 was elected leader, sending a full tablet report...
I20260619 20:48:16.349288 745 ts_tablet_manager.cc:1428] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 20:48:16.349395 730 heartbeater.cc:503] Master 127.13.195.126:45123 was elected leader, sending a full tablet report...
I20260619 20:48:16.350770 748 raft_consensus.cc:348] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.350926 748 raft_consensus.cc:374] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:16.350976 748 raft_consensus.cc:729] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7173eb6f2b34427a9adaac0aca74a329, State: Initialized, Role: FOLLOWER
I20260619 20:48:16.351090 748 consensus_queue.cc:260] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.351357 748 ts_tablet_manager.cc:1428] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 20:48:16.351442 467 heartbeater.cc:503] Master 127.13.195.126:45123 was elected leader, sending a full tablet report...
I20260619 20:48:16.352593 750 raft_consensus.cc:482] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:16.352696 750 raft_consensus.cc:504] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.353021 750 leader_election.cc:288] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7173eb6f2b34427a9adaac0aca74a329 (127.13.195.65:43539), dd6842d031ca4699a5dd2dab1acd30e5 (127.13.195.67:36443)
I20260619 20:48:16.356626 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.353259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:45061 (local address 127.13.195.67:36443)
0619 20:48:16.353306 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.353308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.353319 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:48:16.353343 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.353344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.353353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.353405 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:16.353565 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.354292 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.355718 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.356274 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.356351 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.356359 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.356524 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.356568 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:48:16.356626 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.353308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:55243 (local address 127.13.195.65:43539)
0619 20:48:16.353346 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.353348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.353432 (+ 84us) server_negotiation.cc:407] Connection header received
0619 20:48:16.353514 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.353515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.353526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.353577 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:16.353697 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.354368 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.355718 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.356274 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.356351 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.356359 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.356524 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.356561 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:48:16.356639 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.353162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:36443 (local address 127.13.195.66:45061)
0619 20:48:16.353208 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.353223 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.353295 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.353428 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.353430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.353432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:16.353503 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.353505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.354366 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.354367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.355627 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.355628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.356363 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.356364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.356443 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.356447 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.356505 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.356550 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":28}
I20260619 20:48:16.356647 753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.353216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:43539 (local address 127.13.195.66:55243)
0619 20:48:16.353373 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.353382 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:16.353454 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:16.353579 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:16.353579 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:16.353580 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:16.353638 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.353639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.354454 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.354455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.355625 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:16.355628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.356363 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.356364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:16.356443 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.356447 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:48:16.356505 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:48:16.356548 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":7,"thread_start_us":94,"threads_started":1}
I20260619 20:48:16.356997 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5357b0a3cda14380bfc82b14a453ed80" candidate_uuid: "180694e35ee84817a662d8b6a7e5e97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" is_pre_election: true
I20260619 20:48:16.357139 684 raft_consensus.cc:2454] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 180694e35ee84817a662d8b6a7e5e97b in term 0.
I20260619 20:48:16.357195 421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5357b0a3cda14380bfc82b14a453ed80" candidate_uuid: "180694e35ee84817a662d8b6a7e5e97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7173eb6f2b34427a9adaac0aca74a329" is_pre_election: true
I20260619 20:48:16.357327 421 raft_consensus.cc:2454] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 180694e35ee84817a662d8b6a7e5e97b in term 0.
I20260619 20:48:16.357365 486 leader_election.cc:302] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 180694e35ee84817a662d8b6a7e5e97b, dd6842d031ca4699a5dd2dab1acd30e5; no voters:
I20260619 20:48:16.357460 750 raft_consensus.cc:2781] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:48:16.357513 750 raft_consensus.cc:482] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:48:16.357534 750 raft_consensus.cc:3037] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:16.358201 750 raft_consensus.cc:504] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.358342 750 leader_election.cc:288] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [CANDIDATE]: Term 1 election: Requested vote from peers 7173eb6f2b34427a9adaac0aca74a329 (127.13.195.65:43539), dd6842d031ca4699a5dd2dab1acd30e5 (127.13.195.67:36443)
I20260619 20:48:16.358482 421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5357b0a3cda14380bfc82b14a453ed80" candidate_uuid: "180694e35ee84817a662d8b6a7e5e97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7173eb6f2b34427a9adaac0aca74a329"
I20260619 20:48:16.358551 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5357b0a3cda14380bfc82b14a453ed80" candidate_uuid: "180694e35ee84817a662d8b6a7e5e97b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6842d031ca4699a5dd2dab1acd30e5"
I20260619 20:48:16.358587 421 raft_consensus.cc:3037] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:16.358626 684 raft_consensus.cc:3037] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:16.359447 684 raft_consensus.cc:2454] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 180694e35ee84817a662d8b6a7e5e97b in term 1.
I20260619 20:48:16.359447 421 raft_consensus.cc:2454] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 180694e35ee84817a662d8b6a7e5e97b in term 1.
I20260619 20:48:16.359622 486 leader_election.cc:302] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 180694e35ee84817a662d8b6a7e5e97b, dd6842d031ca4699a5dd2dab1acd30e5; no voters:
I20260619 20:48:16.359736 750 raft_consensus.cc:2781] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:16.359875 750 raft_consensus.cc:686] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 1 LEADER]: Becoming Leader. State: Replica: 180694e35ee84817a662d8b6a7e5e97b, State: Running, Role: LEADER
I20260619 20:48:16.359972 750 consensus_queue.cc:237] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } }
I20260619 20:48:16.360713 32745 catalog_manager.cc:5495] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b reported cstate change: term changed from 0 to 1, leader changed from <none> to 180694e35ee84817a662d8b6a7e5e97b (127.13.195.66). New cstate: current_term: 1 leader_uuid: "180694e35ee84817a662d8b6a7e5e97b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "180694e35ee84817a662d8b6a7e5e97b" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 46451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 } health_report { overall_health: UNKNOWN } } }
I20260619 20:48:16.366761 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.364364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.13.195.66:46451)
0619 20:48:16.364407 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:16.364410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:16.364421 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:48:16.364461 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:16.364462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:16.364478 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:16.364550 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:16.364694 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.365147 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.365913 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:16.366119 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:16.366190 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:16.366338 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:16.366492 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:16.366619 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:16.366649 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 20:48:16.366657 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:48:16.368299 421 raft_consensus.cc:1261] T 5357b0a3cda14380bfc82b14a453ed80 P 7173eb6f2b34427a9adaac0aca74a329 [term 1 FOLLOWER]: Refusing update from remote peer 180694e35ee84817a662d8b6a7e5e97b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:16.368335 684 raft_consensus.cc:1261] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 1 FOLLOWER]: Refusing update from remote peer 180694e35ee84817a662d8b6a7e5e97b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:16.368589 750 consensus_queue.cc:1035] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6842d031ca4699a5dd2dab1acd30e5" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 36443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:16.368705 754 consensus_queue.cc:1035] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7173eb6f2b34427a9adaac0aca74a329" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 43539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:16.371182 757 mvcc.cc:205] Tried to move back new op lower bound from 7298670986722328576 to 7298670986690990080. Current Snapshot: MvccSnapshot[applied={T|T < 7298670986722328576}]
I20260619 20:48:16.371577 759 mvcc.cc:205] Tried to move back new op lower bound from 7298670986722328576 to 7298670986690990080. Current Snapshot: MvccSnapshot[applied={T|T < 7298670986722328576 or (T in {7298670986722328576})}]
I20260619 20:48:16.371757 758 mvcc.cc:205] Tried to move back new op lower bound from 7298670986722328576 to 7298670986690990080. Current Snapshot: MvccSnapshot[applied={T|T < 7298670986722328576}]
I20260619 20:48:16.373746 32745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:16.373828 32745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:16.374611 32745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dedb8c31178741ecacea423a6e6d491f: Sending DeleteTablet for 3 replicas of tablet 5357b0a3cda14380bfc82b14a453ed80
I20260619 20:48:16.374920 533 tablet_service.cc:1514] Processing DeleteTablet for tablet 5357b0a3cda14380bfc82b14a453ed80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:16 UTC) from {username='slave'} at 127.0.0.1:41148
I20260619 20:48:16.375033 664 tablet_service.cc:1514] Processing DeleteTablet for tablet 5357b0a3cda14380bfc82b14a453ed80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:16 UTC) from {username='slave'} at 127.0.0.1:43624
I20260619 20:48:16.375128 766 tablet_replica.cc:330] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: stopping tablet replica
I20260619 20:48:16.375221 766 raft_consensus.cc:2229] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:48:16.375262 400 tablet_service.cc:1514] Processing DeleteTablet for tablet 5357b0a3cda14380bfc82b14a453ed80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:16 UTC) from {username='slave'} at 127.0.0.1:38886
I20260619 20:48:16.375339 766 raft_consensus.cc:2258] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:16.375792 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 338
I20260619 20:48:16.375819 766 ts_tablet_manager.cc:1905] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:16.377579 767 tablet_replica.cc:330] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: stopping tablet replica
I20260619 20:48:16.377691 767 raft_consensus.cc:2229] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:16.377707 766 ts_tablet_manager.cc:1918] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:16.377796 767 raft_consensus.cc:2258] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:16.377882 766 log.cc:1196] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/5357b0a3cda14380bfc82b14a453ed80
I20260619 20:48:16.378212 766 ts_tablet_manager.cc:1939] T 5357b0a3cda14380bfc82b14a453ed80 P 180694e35ee84817a662d8b6a7e5e97b: Deleting consensus metadata
I20260619 20:48:16.378249 767 ts_tablet_manager.cc:1905] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:16.379550 32732 catalog_manager.cc:4841] TS 180694e35ee84817a662d8b6a7e5e97b (127.13.195.66:46451): tablet 5357b0a3cda14380bfc82b14a453ed80 (table test-table [id=d8e5a633964e47ff937995467e0872cf]) successfully deleted
I20260619 20:48:16.379856 767 ts_tablet_manager.cc:1918] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:16.379930 767 log.cc:1196] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/5357b0a3cda14380bfc82b14a453ed80
I20260619 20:48:16.380187 767 ts_tablet_manager.cc:1939] T 5357b0a3cda14380bfc82b14a453ed80 P dd6842d031ca4699a5dd2dab1acd30e5: Deleting consensus metadata
I20260619 20:48:16.380578 32731 catalog_manager.cc:4841] TS dd6842d031ca4699a5dd2dab1acd30e5 (127.13.195.67:36443): tablet 5357b0a3cda14380bfc82b14a453ed80 (table test-table [id=d8e5a633964e47ff937995467e0872cf]) successfully deleted
W20260619 20:48:16.382798 32730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:43539 (error 108)
I20260619 20:48:16.383016 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 470
I20260619 20:48:16.383764 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:16.383611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:43539 (local address 127.0.0.1:38900)
0619 20:48:16.383678 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:16.383723 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:43539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 20:48:16.383895 32730 catalog_manager.cc:4584] TS 7173eb6f2b34427a9adaac0aca74a329 (127.13.195.65:43539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5357b0a3cda14380bfc82b14a453ed80: Network error: Client connection negotiation failed: client connection to 127.13.195.65:43539: connect: Connection refused (error 111)
I20260619 20:48:16.387915 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 602
I20260619 20:48:16.392558 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 32715
2026-06-19T20:48:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (648 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:16 dist-test-slave-n2td krb5kdc[772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:16 dist-test-slave-n2td krb5kdc[772](info): set up 2 sockets
Jun 19 20:48:16 dist-test-slave-n2td krb5kdc[772](info): commencing operation
krb5kdc: starting...
W20260619 20:48:17.437230 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:17 dist-test-slave-n2td krb5kdc[772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:17.555580 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40761
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:40761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:17.639912 786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:17.640128 786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:17.640151 786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:17.641304 786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:17.641345 786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:17.641360 786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:17.641373 786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:17.641425 786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:17.642917 786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:40761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:17.643175 786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:17.643407 786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:17.648386 792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:17.648391 791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:17.648423 786 server_base.cc:1034] running on GCE node
W20260619 20:48:17.648391 795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:17.648950 786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:17.649253 786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:17.650480 786 hybrid_clock.cc:648] HybridClock initialized: now 1781902097650405 us; error 45 us; skew 500 ppm
Jun 19 20:48:17 dist-test-slave-n2td krb5kdc[772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902097, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:17.653362 786 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:17.653853 786 webserver.cc:460] Webserver started at http://127.13.195.126:35141/ using document root <none> and password file <none>
I20260619 20:48:17.654098 786 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:17.654167 786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:17.654301 786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:17.655463 786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "04fb7ae0828b4c1587d07793b837dd2b"
format_stamp: "Formatted at 2026-06-19 20:48:17 on dist-test-slave-n2td"
server_key: "e87349adcb931311d958f753e47ab9d9"
server_key_iv: "a86f08d122afa4e3fb12eba17a786bbe"
server_key_version: "encryptionkey@0"
I20260619 20:48:17.655829 786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "04fb7ae0828b4c1587d07793b837dd2b"
format_stamp: "Formatted at 2026-06-19 20:48:17 on dist-test-slave-n2td"
server_key: "e87349adcb931311d958f753e47ab9d9"
server_key_iv: "a86f08d122afa4e3fb12eba17a786bbe"
server_key_version: "encryptionkey@0"
I20260619 20:48:17.657315 786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:17.658077 802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:17.658267 786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:17.658367 786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "04fb7ae0828b4c1587d07793b837dd2b"
format_stamp: "Formatted at 2026-06-19 20:48:17 on dist-test-slave-n2td"
server_key: "e87349adcb931311d958f753e47ab9d9"
server_key_iv: "a86f08d122afa4e3fb12eba17a786bbe"
server_key_version: "encryptionkey@0"
I20260619 20:48:17.658490 786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:17.694906 786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:17.697288 786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:17.697463 786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:17.701661 786 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:40761
I20260619 20:48:17.701699 857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:40761 every 8 connection(s)
I20260619 20:48:17.702087 786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:17.703011 858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:17.705225 858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b: Bootstrap starting.
I20260619 20:48:17.705761 858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:17.705996 858 log.cc:824] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:17.706746 858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b: No bootstrap required, opened a new log
I20260619 20:48:17.708312 858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fb7ae0828b4c1587d07793b837dd2b" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40761 } }
I20260619 20:48:17.708449 858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:17.708472 858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04fb7ae0828b4c1587d07793b837dd2b, State: Initialized, Role: FOLLOWER
I20260619 20:48:17.708575 858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fb7ae0828b4c1587d07793b837dd2b" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40761 } }
I20260619 20:48:17.708655 858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:17.708688 858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:17.708746 858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:17.709296 858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fb7ae0828b4c1587d07793b837dd2b" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40761 } }
I20260619 20:48:17.709416 858 leader_election.cc:302] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04fb7ae0828b4c1587d07793b837dd2b; no voters:
I20260619 20:48:17.709596 858 leader_election.cc:288] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:17.709635 861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:17.709811 861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [term 1 LEADER]: Becoming Leader. State: Replica: 04fb7ae0828b4c1587d07793b837dd2b, State: Running, Role: LEADER
I20260619 20:48:17.709859 858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:17.709903 861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fb7ae0828b4c1587d07793b837dd2b" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40761 } }
I20260619 20:48:17.710471 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 786
I20260619 20:48:17.710616 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:17.710702 862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04fb7ae0828b4c1587d07793b837dd2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fb7ae0828b4c1587d07793b837dd2b" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40761 } } }
I20260619 20:48:17.710789 862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:17.710850 14093 external_mini_cluster.cc:1428] Setting key c2596387e1b9393bf372dd79ce5093f3
I20260619 20:48:17.711229 871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:17.711669 863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04fb7ae0828b4c1587d07793b837dd2b. Latest consensus state: current_term: 1 leader_uuid: "04fb7ae0828b4c1587d07793b837dd2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fb7ae0828b4c1587d07793b837dd2b" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40761 } } }
I20260619 20:48:17.711786 871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:17.711817 863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:17.713286 871 catalog_manager.cc:1347] Generated new cluster ID: a1227b95ebfe46f4b24bb708b90883ff
I20260619 20:48:17.713337 871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:17 dist-test-slave-n2td krb5kdc[772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:17.722899 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:17.711475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.13.195.126:40761)
0619 20:48:17.711650 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:17.711659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:17.711913 (+ 254us) server_negotiation.cc:407] Connection header received
0619 20:48:17.712030 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:17.712037 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:17.712102 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:17.712240 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:17.713025 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:17.714448 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:17.715452 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:17.716094 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:17.716217 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:17.719086 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:17.719090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:17.719090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:17.719103 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:17.721072 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:17.721483 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:17.721486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:17.721486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:17.721615 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:17.721832 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:17.721833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:17.721833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:17.722133 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:17.722265 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:17.722394 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 20:48:17.722665 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260619 20:48:17.727257 871 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:17.727731 871 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:17.732384 871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b: Generated new TSK 0
I20260619 20:48:17.732584 871 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.195.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
Entry for principal kudu/127.13.195.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab.
I20260619 20:48:17.763478 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40761
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:17.846638 885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:17.846845 885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:17.846870 885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:17.848060 885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:17.848104 885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:17.848131 885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:48:17.849673 885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.65.keytab
--principal=kudu/127.13.195.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:17.849931 885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:17.850165 885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:17.855311 891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:17.855289 890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:17.855283 893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:17.855705 885 server_base.cc:1034] running on GCE node
I20260619 20:48:17.855935 885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:17.856180 885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:17.857327 885 hybrid_clock.cc:648] HybridClock initialized: now 1781902097857315 us; error 29 us; skew 500 ppm
Jun 19 20:48:17 dist-test-slave-n2td krb5kdc[772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902097, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:17.859890 885 init.cc:377] Logged in from keytab as kudu/127.13.195.65@KRBTEST.COM (short username kudu)
I20260619 20:48:17.860368 885 webserver.cc:460] Webserver started at http://127.13.195.65:38511/ using document root <none> and password file <none>
I20260619 20:48:17.860608 885 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:17.860656 885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:17.860786 885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:17.861943 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520"
format_stamp: "Formatted at 2026-06-19 20:48:17 on dist-test-slave-n2td"
server_key: "281821ca060e620d21e916f7829e6a25"
server_key_iv: "70fc5e437b67d2f69862e60aa7db0fc9"
server_key_version: "encryptionkey@0"
I20260619 20:48:17.862334 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520"
format_stamp: "Formatted at 2026-06-19 20:48:17 on dist-test-slave-n2td"
server_key: "281821ca060e620d21e916f7829e6a25"
server_key_iv: "70fc5e437b67d2f69862e60aa7db0fc9"
server_key_version: "encryptionkey@0"
I20260619 20:48:17.863715 885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:17.864568 900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:17.864775 885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:48:17.864884 885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520"
format_stamp: "Formatted at 2026-06-19 20:48:17 on dist-test-slave-n2td"
server_key: "281821ca060e620d21e916f7829e6a25"
server_key_iv: "70fc5e437b67d2f69862e60aa7db0fc9"
server_key_version: "encryptionkey@0"
I20260619 20:48:17.864961 885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:17.876506 885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:17.878837 885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:17.879033 885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:17.879302 885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:17.879693 885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:17.879757 885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:17.879796 885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:17.879841 885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:17.886144 885 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:40607
I20260619 20:48:17.886194 1015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:40607 every 8 connection(s)
I20260619 20:48:17.886628 885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:48:17.888913 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 885
I20260619 20:48:17.889015 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:48:17.889223 14093 external_mini_cluster.cc:1428] Setting key 02320be02c2448270bc33cdda8b4400f
Jun 19 20:48:17 dist-test-slave-n2td krb5kdc[772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902097, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.65@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:17.897984 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:17.887354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:54457 (local address 127.13.195.126:40761)
0619 20:48:17.887427 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:17.887429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:17.887712 (+ 283us) server_negotiation.cc:407] Connection header received
0619 20:48:17.888281 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:17.888282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:17.888297 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:17.888350 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:17.889622 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:17.890321 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:17.891725 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:17.891931 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:17.891990 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:17.894746 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:17.894748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:17.894748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:17.894757 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:17.896467 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:17.896884 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:17.896885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:17.896886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:17.896951 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:17.897221 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:17.897222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:17.897222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:17.897374 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:17.897447 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:17.897734 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 20:48:17.897835 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:48:17.898018 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:17.887357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40761 (local address 127.13.195.65:54457)
0619 20:48:17.887575 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:17.887615 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:17.888183 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:17.888433 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:17.888437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:17.889180 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:17.889516 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:17.889524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:17.890418 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:17.890419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:17.891623 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:17.891626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:17.892033 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:17.892034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:17.892207 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:17.892846 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:17.892855 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:17.894644 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:17.896556 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:17.896563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:17.896565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:17.896802 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:17.897030 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:17.897031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:17.897031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:17.897148 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:17.897444 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:17.897448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:17.897590 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 20:48:17.897684 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 20:48:17.897818 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.13.195.66@KRBTEST.COM; defaulting to no policy
I20260619 20:48:17.898671 1016 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40761
I20260619 20:48:17.898797 1016 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:17.898999 1016 heartbeater.cc:511] Master 127.13.195.126:40761 requested a full tablet report, sending...
I20260619 20:48:17.899602 821 ts_manager.cc:194] Registered new tserver with Master: c5d1fbb8997a4e0fa391adaaa9b4e520 (127.13.195.65:40607)
I20260619 20:48:17.900307 821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.65@KRBTEST.COM'} at 127.13.195.65:54457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
Entry for principal kudu/127.13.195.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab.
I20260619 20:48:17.917469 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40761
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:17.998513 1021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:17.998764 1021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:17.998802 1021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:17.999953 1021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:18.000029 1021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:18.000074 1021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:48:18.001534 1021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.66.keytab
--principal=kudu/127.13.195.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:18.001852 1021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:18.002094 1021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:18.007422 1026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.007560 1027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.007417 1029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.007611 1021 server_base.cc:1034] running on GCE node
I20260619 20:48:18.007900 1021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:18.008159 1021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:18.009341 1021 hybrid_clock.cc:648] HybridClock initialized: now 1781902098009317 us; error 35 us; skew 500 ppm
Jun 19 20:48:18 dist-test-slave-n2td krb5kdc[772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902098, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:18.012189 1021 init.cc:377] Logged in from keytab as kudu/127.13.195.66@KRBTEST.COM (short username kudu)
I20260619 20:48:18.012585 1021 webserver.cc:460] Webserver started at http://127.13.195.66:33595/ using document root <none> and password file <none>
I20260619 20:48:18.012835 1021 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:18.012882 1021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:18.013029 1021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:18.014077 1021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "c59bca9948454f5c97fcfb916ae786a7"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "86f8fc03567c81b0c25f2bb673d9cd32"
server_key_iv: "d5388e63055cfd7cb3e3c7cc09800f6c"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.014482 1021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "c59bca9948454f5c97fcfb916ae786a7"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "86f8fc03567c81b0c25f2bb673d9cd32"
server_key_iv: "d5388e63055cfd7cb3e3c7cc09800f6c"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.015836 1021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:18.016589 1036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.016752 1021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:18.016851 1021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "c59bca9948454f5c97fcfb916ae786a7"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "86f8fc03567c81b0c25f2bb673d9cd32"
server_key_iv: "d5388e63055cfd7cb3e3c7cc09800f6c"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.016932 1021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:18.032482 1021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:18.034898 1021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:18.035079 1021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:18.035331 1021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:18.035660 1021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:18.035714 1021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.035758 1021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:18.035786 1021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.041754 1021 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:44111
I20260619 20:48:18.041812 1150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:44111 every 8 connection(s)
I20260619 20:48:18.042196 1021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:48:18.042522 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1021
I20260619 20:48:18.042675 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:48:18.042903 14093 external_mini_cluster.cc:1428] Setting key acd2d6297c56ab9ae875019c59f3e718
Jun 19 20:48:18 dist-test-slave-n2td krb5kdc[772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902098, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.66@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.67@KRBTEST.COM; defaulting to no policy
I20260619 20:48:18.053607 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.043085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:58927 (local address 127.13.195.126:40761)
0619 20:48:18.043141 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.043144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.043407 (+ 263us) server_negotiation.cc:407] Connection header received
0619 20:48:18.044058 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.044058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.044069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.044107 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.045448 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.046256 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.047343 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.047692 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.047777 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.050338 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.050340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.050340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:18.050349 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.052039 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:18.052485 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:18.052487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:18.052487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:18.052557 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:18.052878 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:18.052879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:18.052879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:18.053034 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.053101 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.053419 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.053538 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:48:18.053790 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.043045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40761 (local address 127.13.195.66:58927)
0619 20:48:18.043257 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.043302 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.043951 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.044181 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.044185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.045034 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:18.045337 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.045345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.046355 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.046356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.047246 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.047249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.047786 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.047787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.047905 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.048517 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:18.048526 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:18.050241 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:18.052136 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:18.052142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:18.052144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:18.052413 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:18.052644 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:18.052645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:18.052645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:18.052787 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:18.053146 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:18.053152 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:18.053284 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.053376 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.053531 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260619 20:48:18.054348 1151 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40761
I20260619 20:48:18.054489 1151 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.67@KRBTEST.COM" created.
I20260619 20:48:18.054699 1151 heartbeater.cc:511] Master 127.13.195.126:40761 requested a full tablet report, sending...
I20260619 20:48:18.055153 821 ts_manager.cc:194] Registered new tserver with Master: c59bca9948454f5c97fcfb916ae786a7 (127.13.195.66:44111)
I20260619 20:48:18.055541 821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.66@KRBTEST.COM'} at 127.13.195.66:58927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
Entry for principal kudu/127.13.195.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab.
I20260619 20:48:18.069959 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40761
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:18.150828 1156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:18.151111 1156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:18.151154 1156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:18.152297 1156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:18.152369 1156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:18.152415 1156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:48:18.153859 1156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.67.keytab
--principal=kudu/127.13.195.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:18.154134 1156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:18.154364 1156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:48:18.159379 1156 server_base.cc:1034] running on GCE node
W20260619 20:48:18.159341 1162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.159365 1161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.159341 1164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.159760 1156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:18.159965 1156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:18.161106 1156 hybrid_clock.cc:648] HybridClock initialized: now 1781902098161096 us; error 30 us; skew 500 ppm
Jun 19 20:48:18 dist-test-slave-n2td krb5kdc[772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902098, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:18.163667 1156 init.cc:377] Logged in from keytab as kudu/127.13.195.67@KRBTEST.COM (short username kudu)
I20260619 20:48:18.164110 1156 webserver.cc:460] Webserver started at http://127.13.195.67:46037/ using document root <none> and password file <none>
I20260619 20:48:18.164358 1156 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:18.164422 1156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:18.164564 1156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:18.165565 1156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "e58c5dd5e79b44449d5d4791b65eb8b3"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "9a46b1a4c692e8042c6731e45910cf46"
server_key_iv: "0f149fd92f9985970ea8df77fdb5d556"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.165894 1156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "e58c5dd5e79b44449d5d4791b65eb8b3"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "9a46b1a4c692e8042c6731e45910cf46"
server_key_iv: "0f149fd92f9985970ea8df77fdb5d556"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.167266 1156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:48:18.168232 1171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.168404 1156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:18.168511 1156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "e58c5dd5e79b44449d5d4791b65eb8b3"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "9a46b1a4c692e8042c6731e45910cf46"
server_key_iv: "0f149fd92f9985970ea8df77fdb5d556"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.168582 1156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:18.180968 1156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:18.182866 1156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:18.183027 1156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:18.183260 1156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:18.183559 1156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:18.183609 1156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.183643 1156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:18.183671 1156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.189647 1156 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:34983
I20260619 20:48:18.189688 1285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:34983 every 8 connection(s)
I20260619 20:48:18.190029 1156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:48:18.194973 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1156
I20260619 20:48:18.195111 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:48:18.195309 14093 external_mini_cluster.cc:1428] Setting key b06c9b8eecb8c22e064d1bce733ae56c
Jun 19 20:48:18 dist-test-slave-n2td krb5kdc[772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902098, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.67@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:18.199213 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.190768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:39885 (local address 127.13.195.126:40761)
0619 20:48:18.190814 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.190816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.191024 (+ 208us) server_negotiation.cc:407] Connection header received
0619 20:48:18.191500 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.191501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.191510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.191564 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.192477 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.192959 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.193895 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.194070 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.194128 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.196639 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.196640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.196641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:18.196646 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.198018 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:18.198410 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:18.198411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:18.198412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:18.198489 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:18.198702 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:18.198703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:18.198703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:18.198805 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.198855 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.199089 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.199150 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:48:18.199438 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.190745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40761 (local address 127.13.195.67:39885)
0619 20:48:18.190907 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.190948 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.191414 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.191629 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.191632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.192176 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:18.192391 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.192396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.193035 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.193036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.193823 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.193825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.194140 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.194141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.194241 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.194838 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:18.194849 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:18.196559 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:18.198086 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:18.198090 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:18.198092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:18.198304 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:18.198553 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:18.198553 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:18.198553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:18.198651 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:18.198861 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:18.198865 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:18.198978 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.199059 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.199185 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260619 20:48:18.199972 1286 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40761
I20260619 20:48:18.200091 1286 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:18.200266 1286 heartbeater.cc:511] Master 127.13.195.126:40761 requested a full tablet report, sending...
I20260619 20:48:18.200631 821 ts_manager.cc:194] Registered new tserver with Master: e58c5dd5e79b44449d5d4791b65eb8b3 (127.13.195.67:34983)
I20260619 20:48:18.201012 821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.195.67@KRBTEST.COM'} at 127.13.195.67:39885
I20260619 20:48:18.207266 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:48:18.214681 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.208597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.13.195.126:40761)
0619 20:48:18.208680 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.208682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.208739 (+ 57us) server_negotiation.cc:407] Connection header received
0619 20:48:18.208779 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.208780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.208788 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.208833 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.209347 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.209903 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.210706 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.210883 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.210936 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.212214 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.212215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.212216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:18.212222 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.213676 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:18.213958 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:18.213958 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:18.213959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:18.214001 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:18.214143 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:18.214143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:18.214143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:18.214268 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.214338 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.214481 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.214565 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:48:18.216149 821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:48:18.217752 821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:48:18.224073 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.220027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:44111 (local address 127.0.0.1:54974)
0619 20:48:18.220238 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.220282 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.220350 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.220596 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.220599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.220602 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.220731 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.220733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.221793 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.221794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.222854 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.222857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.223763 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.223764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.223846 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.223860 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.223936 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.223988 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
I20260619 20:48:18.224084 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.220209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.13.195.66:44111)
0619 20:48:18.220391 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.220396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.220430 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:48:18.220476 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.220478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.220493 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.220577 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.220820 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.221685 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.222941 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.223681 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.223752 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.223815 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.223960 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.224000 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":65,"threads_started":1}
I20260619 20:48:18.224740 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.220028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:34983 (local address 127.0.0.1:60156)
0619 20:48:18.220355 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.220364 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.220402 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.221120 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.221121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.221123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.221164 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.221166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.222452 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.222453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.223613 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.223616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.224495 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.224496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.224566 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.224569 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.224626 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.224665 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":30,"thread_start_us":95,"threads_started":1}
I20260619 20:48:18.224751 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.220097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60156 (local address 127.13.195.67:34983)
0619 20:48:18.220937 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.220942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.220952 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:18.220988 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.220990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.221004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.221084 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.221223 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.222307 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.223712 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.224428 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.224496 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.224545 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.224640 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.224675 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":89,"threads_started":1}
I20260619 20:48:18.224907 1085 tablet_service.cc:1467] Processing CreateTablet for tablet 8a898262be364853bdbbfbab4014e334 (DEFAULT_TABLE table=test-table [id=718404a735c2480390c934fe4d1c864f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:18.225299 1085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a898262be364853bdbbfbab4014e334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:18.225342 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.220264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:40607 (local address 127.0.0.1:42830)
0619 20:48:18.220689 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.220699 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.220787 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.221053 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.221054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.221057 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.221113 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.221115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.222423 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.222424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.223472 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.223474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.225054 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.225055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.225166 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.225171 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.225220 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.225264 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":48,"threads_started":1}
I20260619 20:48:18.225550 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.220377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.13.195.65:40607)
0619 20:48:18.220603 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.220608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.220770 (+ 162us) server_negotiation.cc:407] Connection header received
0619 20:48:18.220871 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.220875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.220893 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.220969 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.221188 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.222307 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.223562 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.224663 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.224732 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.224783 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.225429 (+ 646us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.225476 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260619 20:48:18.227190 949 tablet_service.cc:1467] Processing CreateTablet for tablet 8a898262be364853bdbbfbab4014e334 (DEFAULT_TABLE table=test-table [id=718404a735c2480390c934fe4d1c864f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:18.227190 1220 tablet_service.cc:1467] Processing CreateTablet for tablet 8a898262be364853bdbbfbab4014e334 (DEFAULT_TABLE table=test-table [id=718404a735c2480390c934fe4d1c864f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:18.227523 949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a898262be364853bdbbfbab4014e334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:18.227525 1220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a898262be364853bdbbfbab4014e334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:18.228142 1300 tablet_bootstrap.cc:492] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Bootstrap starting.
I20260619 20:48:18.228760 1300 tablet_bootstrap.cc:654] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:18.229189 1300 log.cc:824] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:18.229848 1300 tablet_bootstrap.cc:492] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: No bootstrap required, opened a new log
I20260619 20:48:18.229892 1300 ts_tablet_manager.cc:1397] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:48:18.230805 1303 tablet_bootstrap.cc:492] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Bootstrap starting.
I20260619 20:48:18.231442 1303 tablet_bootstrap.cc:654] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:18.231765 1300 raft_consensus.cc:348] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.231994 1300 raft_consensus.cc:374] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:18.232033 1300 raft_consensus.cc:729] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59bca9948454f5c97fcfb916ae786a7, State: Initialized, Role: FOLLOWER
I20260619 20:48:18.232017 1303 log.cc:824] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:18.232174 1300 consensus_queue.cc:260] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.232366 1302 tablet_bootstrap.cc:492] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520: Bootstrap starting.
I20260619 20:48:18.232465 1300 ts_tablet_manager.cc:1428] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:48:18.232599 1151 heartbeater.cc:503] Master 127.13.195.126:40761 was elected leader, sending a full tablet report...
I20260619 20:48:18.232885 1303 tablet_bootstrap.cc:492] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: No bootstrap required, opened a new log
I20260619 20:48:18.232952 1303 ts_tablet_manager.cc:1397] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:48:18.233050 1302 tablet_bootstrap.cc:654] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:18.233631 1302 log.cc:824] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:18.234370 1302 tablet_bootstrap.cc:492] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520: No bootstrap required, opened a new log
I20260619 20:48:18.234476 1302 ts_tablet_manager.cc:1397] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:48:18.234798 1303 raft_consensus.cc:348] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.234932 1303 raft_consensus.cc:374] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:18.234954 1303 raft_consensus.cc:729] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58c5dd5e79b44449d5d4791b65eb8b3, State: Initialized, Role: FOLLOWER
I20260619 20:48:18.235060 1303 consensus_queue.cc:260] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.235338 1303 ts_tablet_manager.cc:1428] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:48:18.235358 1286 heartbeater.cc:503] Master 127.13.195.126:40761 was elected leader, sending a full tablet report...
I20260619 20:48:18.236483 1302 raft_consensus.cc:348] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.236660 1302 raft_consensus.cc:374] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:18.236706 1302 raft_consensus.cc:729] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d1fbb8997a4e0fa391adaaa9b4e520, State: Initialized, Role: FOLLOWER
I20260619 20:48:18.236865 1302 consensus_queue.cc:260] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.236979 1308 raft_consensus.cc:482] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:18.237069 1308 raft_consensus.cc:504] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.237412 1302 ts_tablet_manager.cc:1428] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:48:18.237422 1016 heartbeater.cc:503] Master 127.13.195.126:40761 was elected leader, sending a full tablet report...
I20260619 20:48:18.237815 1308 leader_election.cc:288] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e58c5dd5e79b44449d5d4791b65eb8b3 (127.13.195.67:34983), c59bca9948454f5c97fcfb916ae786a7 (127.13.195.66:44111)
I20260619 20:48:18.240460 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.237555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:34983 (local address 127.13.195.65:49531)
0619 20:48:18.237681 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.237694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.237752 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.238026 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.238027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.238029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.238093 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.238094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.238900 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.238902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.239627 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.239629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.240194 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.240195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.240272 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.240282 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.240355 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.240394 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260619 20:48:18.240460 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.237646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:49531 (local address 127.13.195.67:34983)
0619 20:48:18.237684 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.237687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.237737 (+ 50us) server_negotiation.cc:407] Connection header received
0619 20:48:18.237841 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.237842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.237853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.237916 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.238169 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.238829 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.239731 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.240103 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.240194 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.240202 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.240362 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.240396 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:18.240600 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.237636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:36129 (local address 127.13.195.66:44111)
0619 20:48:18.237677 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.237679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.237908 (+ 229us) server_negotiation.cc:407] Connection header received
0619 20:48:18.237967 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.237967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.237978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.238033 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.238193 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.238903 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.239836 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.240215 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.240309 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.240316 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.240474 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.240511 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:18.240789 1240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a898262be364853bdbbfbab4014e334" candidate_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" is_pre_election: true
I20260619 20:48:18.240887 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.237560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:44111 (local address 127.13.195.65:36129)
0619 20:48:18.237852 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.237863 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.237919 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.238037 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.238038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.238040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.238126 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.238127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.238980 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.238981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.239726 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.239727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.240310 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.240311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.240381 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.240385 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.240468 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.240809 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":138,"thread_start_us":73,"threads_started":1}
I20260619 20:48:18.240921 1240 raft_consensus.cc:2454] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5d1fbb8997a4e0fa391adaaa9b4e520 in term 0.
I20260619 20:48:18.241123 903 leader_election.cc:302] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5d1fbb8997a4e0fa391adaaa9b4e520, e58c5dd5e79b44449d5d4791b65eb8b3; no voters:
I20260619 20:48:18.241247 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a898262be364853bdbbfbab4014e334" candidate_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59bca9948454f5c97fcfb916ae786a7" is_pre_election: true
I20260619 20:48:18.241360 1105 raft_consensus.cc:2454] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5d1fbb8997a4e0fa391adaaa9b4e520 in term 0.
I20260619 20:48:18.241436 1308 raft_consensus.cc:2781] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:48:18.241477 1308 raft_consensus.cc:482] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:48:18.241492 1308 raft_consensus.cc:3037] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:18.242117 1308 raft_consensus.cc:504] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.242308 1308 leader_election.cc:288] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [CANDIDATE]: Term 1 election: Requested vote from peers e58c5dd5e79b44449d5d4791b65eb8b3 (127.13.195.67:34983), c59bca9948454f5c97fcfb916ae786a7 (127.13.195.66:44111)
I20260619 20:48:18.242368 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a898262be364853bdbbfbab4014e334" candidate_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59bca9948454f5c97fcfb916ae786a7"
I20260619 20:48:18.242476 1105 raft_consensus.cc:3037] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:18.242501 1240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a898262be364853bdbbfbab4014e334" candidate_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3"
I20260619 20:48:18.242571 1240 raft_consensus.cc:3037] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:18.243372 1240 raft_consensus.cc:2454] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5d1fbb8997a4e0fa391adaaa9b4e520 in term 1.
I20260619 20:48:18.243372 1105 raft_consensus.cc:2454] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5d1fbb8997a4e0fa391adaaa9b4e520 in term 1.
I20260619 20:48:18.243541 902 leader_election.cc:302] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c59bca9948454f5c97fcfb916ae786a7, c5d1fbb8997a4e0fa391adaaa9b4e520; no voters:
I20260619 20:48:18.243672 1308 raft_consensus.cc:2781] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:18.243747 1308 raft_consensus.cc:686] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 1 LEADER]: Becoming Leader. State: Replica: c5d1fbb8997a4e0fa391adaaa9b4e520, State: Running, Role: LEADER
I20260619 20:48:18.243827 1308 consensus_queue.cc:237] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } }
I20260619 20:48:18.244637 821 catalog_manager.cc:5495] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5d1fbb8997a4e0fa391adaaa9b4e520 (127.13.195.65). New cstate: current_term: 1 leader_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5d1fbb8997a4e0fa391adaaa9b4e520" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 40607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 } health_report { overall_health: UNKNOWN } } }
I20260619 20:48:18.254304 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.251985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.13.195.65:40607)
0619 20:48:18.252048 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.252049 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.252059 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:18.252103 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.252104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.252120 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.252172 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:18.252295 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.252763 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.253480 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.253677 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.253735 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.253854 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:18.253949 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:18.254003 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.254049 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.254069 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:48:18.256091 1105 raft_consensus.cc:1261] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 1 FOLLOWER]: Refusing update from remote peer c5d1fbb8997a4e0fa391adaaa9b4e520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:18.256091 1240 raft_consensus.cc:1261] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 1 FOLLOWER]: Refusing update from remote peer c5d1fbb8997a4e0fa391adaaa9b4e520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:18.256475 1309 consensus_queue.cc:1035] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59bca9948454f5c97fcfb916ae786a7" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 44111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:18.256645 1308 consensus_queue.cc:1035] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58c5dd5e79b44449d5d4791b65eb8b3" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:18.259673 1316 mvcc.cc:205] Tried to move back new op lower bound from 7298670994454118400 to 7298670994407321600. Current Snapshot: MvccSnapshot[applied={T|T < 7298670994454118400}]
I20260619 20:48:18.261372 821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:18.261461 821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:18.262172 821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04fb7ae0828b4c1587d07793b837dd2b: Sending DeleteTablet for 3 replicas of tablet 8a898262be364853bdbbfbab4014e334
I20260619 20:48:18.262514 1085 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a898262be364853bdbbfbab4014e334 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:18 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:54974
I20260619 20:48:18.262518 949 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a898262be364853bdbbfbab4014e334 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:18 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:42830
I20260619 20:48:18.262744 1324 tablet_replica.cc:330] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: stopping tablet replica
I20260619 20:48:18.262846 1324 raft_consensus.cc:2229] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:18.262871 1220 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a898262be364853bdbbfbab4014e334 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:18 UTC) from {username='kudu', principal='kudu/127.13.195.126@KRBTEST.COM'} at 127.0.0.1:60156
I20260619 20:48:18.262943 1324 raft_consensus.cc:2258] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:18.263043 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 885
I20260619 20:48:18.263072 1323 tablet_replica.cc:330] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520: stopping tablet replica
I20260619 20:48:18.263152 1323 raft_consensus.cc:2229] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:48:18.263227 1323 raft_consensus.cc:2258] T 8a898262be364853bdbbfbab4014e334 P c5d1fbb8997a4e0fa391adaaa9b4e520 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:18.263734 1325 tablet_replica.cc:330] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: stopping tablet replica
I20260619 20:48:18.263820 1325 raft_consensus.cc:2229] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:18.263906 1325 raft_consensus.cc:2258] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:18.263926 1324 ts_tablet_manager.cc:1905] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:18.264312 1325 ts_tablet_manager.cc:1905] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:18.265610 1324 ts_tablet_manager.cc:1918] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:18.265686 1324 log.cc:1196] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/8a898262be364853bdbbfbab4014e334
I20260619 20:48:18.265918 1325 ts_tablet_manager.cc:1918] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:18.265972 1324 ts_tablet_manager.cc:1939] T 8a898262be364853bdbbfbab4014e334 P c59bca9948454f5c97fcfb916ae786a7: Deleting consensus metadata
I20260619 20:48:18.265986 1325 log.cc:1196] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/8a898262be364853bdbbfbab4014e334
I20260619 20:48:18.266230 1325 ts_tablet_manager.cc:1939] T 8a898262be364853bdbbfbab4014e334 P e58c5dd5e79b44449d5d4791b65eb8b3: Deleting consensus metadata
I20260619 20:48:18.266422 804 catalog_manager.cc:4841] TS c59bca9948454f5c97fcfb916ae786a7 (127.13.195.66:44111): tablet 8a898262be364853bdbbfbab4014e334 (table test-table [id=718404a735c2480390c934fe4d1c864f]) successfully deleted
I20260619 20:48:18.266613 805 catalog_manager.cc:4841] TS e58c5dd5e79b44449d5d4791b65eb8b3 (127.13.195.67:34983): tablet 8a898262be364853bdbbfbab4014e334 (table test-table [id=718404a735c2480390c934fe4d1c864f]) successfully deleted
W20260619 20:48:18.269462 805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:40607 (error 108)
I20260619 20:48:18.269992 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.269846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:40607 (local address 127.0.0.1:42852)
0619 20:48:18.269901 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.269953 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:40607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:48:18.270011 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1021
W20260619 20:48:18.270133 805 catalog_manager.cc:4584] TS c5d1fbb8997a4e0fa391adaaa9b4e520 (127.13.195.65:40607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a898262be364853bdbbfbab4014e334: Network error: Client connection negotiation failed: client connection to 127.13.195.65:40607: connect: Connection refused (error 111)
I20260619 20:48:18.275399 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1156
I20260619 20:48:18.279738 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 786
2026-06-19T20:48:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1887 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T20:48:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:18Z Disabled control of system clock
I20260619 20:48:18.307248 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40619
--webserver_interface=127.13.195.126
--webserver_port=0
--builtin_ntp_servers=127.13.195.84:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:40619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:18.389776 1330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:18.390007 1330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:18.390028 1330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:18.391553 1330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:18.391608 1330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:18.391621 1330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:18.391635 1330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:18.391647 1330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:18.393134 1330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:40619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:40619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:18.393352 1330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:18.393613 1330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:18.398473 1335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.398422 1338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.398633 1330 server_base.cc:1034] running on GCE node
W20260619 20:48:18.398501 1336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.398991 1330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:18.399241 1330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:18.400393 1330 hybrid_clock.cc:648] HybridClock initialized: now 1781902098400370 us; error 42 us; skew 500 ppm
I20260619 20:48:18.401794 1330 webserver.cc:460] Webserver started at http://127.13.195.126:38227/ using document root <none> and password file <none>
I20260619 20:48:18.402055 1330 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:18.402104 1330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:18.402264 1330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:18.403409 1330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "6cb96d3bf9284d48808b03af61ca6621"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "2f53af024c843b09290b4e4cf23a96a8"
server_key_iv: "169b836df0bc4750649d86288e109ae0"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.403848 1330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "6cb96d3bf9284d48808b03af61ca6621"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "2f53af024c843b09290b4e4cf23a96a8"
server_key_iv: "169b836df0bc4750649d86288e109ae0"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.405182 1330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:18.406083 1344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.406489 1330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:18.406611 1330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "6cb96d3bf9284d48808b03af61ca6621"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "2f53af024c843b09290b4e4cf23a96a8"
server_key_iv: "169b836df0bc4750649d86288e109ae0"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.406704 1330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:18.418828 1330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:18.419178 1330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:18.419341 1330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:18.423543 1330 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:40619
I20260619 20:48:18.423586 1396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:40619 every 8 connection(s)
I20260619 20:48:18.424022 1330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:18.425009 1397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:18.427699 1397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621: Bootstrap starting.
I20260619 20:48:18.428283 1397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:18.428552 1397 log.cc:824] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:18.429217 1397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621: No bootstrap required, opened a new log
I20260619 20:48:18.430893 1397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb96d3bf9284d48808b03af61ca6621" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40619 } }
I20260619 20:48:18.431030 1397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:18.431052 1397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb96d3bf9284d48808b03af61ca6621, State: Initialized, Role: FOLLOWER
I20260619 20:48:18.431195 1397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb96d3bf9284d48808b03af61ca6621" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40619 } }
I20260619 20:48:18.431278 1397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:18.431303 1397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:18.431330 1397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:18.431665 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1330
I20260619 20:48:18.431797 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:18.431877 1397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb96d3bf9284d48808b03af61ca6621" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40619 } }
I20260619 20:48:18.432001 1397 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cb96d3bf9284d48808b03af61ca6621; no voters:
I20260619 20:48:18.432024 14093 external_mini_cluster.cc:1428] Setting key 0579852866ae112303216466d810bc82
I20260619 20:48:18.432179 1397 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:18.432238 1400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:18.432407 1400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [term 1 LEADER]: Becoming Leader. State: Replica: 6cb96d3bf9284d48808b03af61ca6621, State: Running, Role: LEADER
I20260619 20:48:18.432469 1397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:18.432494 1400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb96d3bf9284d48808b03af61ca6621" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40619 } }
I20260619 20:48:18.433027 1401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cb96d3bf9284d48808b03af61ca6621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb96d3bf9284d48808b03af61ca6621" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40619 } } }
I20260619 20:48:18.433125 1401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:18.433270 1402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb96d3bf9284d48808b03af61ca6621. Latest consensus state: current_term: 1 leader_uuid: "6cb96d3bf9284d48808b03af61ca6621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb96d3bf9284d48808b03af61ca6621" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 40619 } } }
I20260619 20:48:18.433316 1411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:18.433333 1402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:18.434013 1411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:18.435775 1411 catalog_manager.cc:1347] Generated new cluster ID: a9c0ee87aa1b42488475c9f5ac90a530
I20260619 20:48:18.435829 1411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:18.438995 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.432810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.13.195.126:40619)
0619 20:48:18.432996 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.433008 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.433045 (+ 37us) server_negotiation.cc:407] Connection header received
0619 20:48:18.433646 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.433653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.433727 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.433846 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.434223 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.435480 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.436572 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.437218 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.437349 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.438241 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.438245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.438246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:18.438487 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.438532 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:18.438556 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.438632 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.438702 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.438787 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260619 20:48:18.451924 1411 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:18.452651 1411 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:18.458042 1411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621: Generated new TSK 0
I20260619 20:48:18.458240 1411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:18.465793 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.65:0
--local_ip_for_outbound_sockets=127.13.195.65
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40619
--builtin_ntp_servers=127.13.195.84:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:18.547832 1421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:18.548035 1421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:18.548065 1421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:18.549253 1421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:18.549297 1421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:18.549324 1421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.65
I20260619 20:48:18.550760 1421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.195.65
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.65
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:18.550994 1421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:18.551314 1421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:18.556195 1426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.556278 1429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.556195 1427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.556519 1421 server_base.cc:1034] running on GCE node
I20260619 20:48:18.556741 1421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:18.556980 1421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:18.558141 1421 hybrid_clock.cc:648] HybridClock initialized: now 1781902098558119 us; error 36 us; skew 500 ppm
I20260619 20:48:18.559445 1421 webserver.cc:460] Webserver started at http://127.13.195.65:34885/ using document root <none> and password file <none>
I20260619 20:48:18.559695 1421 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:18.559763 1421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:18.559912 1421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:18.560981 1421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data/instance:
uuid: "b532160676c748b7a197f1aad5920161"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "314e0500e279d1f63d8ad511435da380"
server_key_iv: "f77a1c9810dafebca2d4b30e282ed59e"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.561350 1421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance:
uuid: "b532160676c748b7a197f1aad5920161"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "314e0500e279d1f63d8ad511435da380"
server_key_iv: "f77a1c9810dafebca2d4b30e282ed59e"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.562726 1421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:18.563573 1435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.563808 1421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:18.563936 1421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/wal
uuid: "b532160676c748b7a197f1aad5920161"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "314e0500e279d1f63d8ad511435da380"
server_key_iv: "f77a1c9810dafebca2d4b30e282ed59e"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.564023 1421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:18.576639 1421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:18.576993 1421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:18.577158 1421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:18.577414 1421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:18.577755 1421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:18.577809 1421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.577854 1421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:18.577880 1421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.584035 1421 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.65:38783
I20260619 20:48:18.584093 1549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.65:38783 every 8 connection(s)
I20260619 20:48:18.584414 1421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/data/info.pb
I20260619 20:48:18.589627 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.585128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.65:34363 (local address 127.13.195.126:40619)
0619 20:48:18.585239 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.585241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.585437 (+ 196us) server_negotiation.cc:407] Connection header received
0619 20:48:18.585922 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.585923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.585936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.585992 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.586716 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.587220 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.588163 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.588359 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.588412 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.589191 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.589192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.589192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:18.589197 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.589211 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:18.589215 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.589272 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.589426 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.589527 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 20:48:18.589694 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.585093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40619 (local address 127.13.195.65:34363)
0619 20:48:18.585314 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.585350 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.585833 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.586048 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.586057 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.586066 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:18.586642 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.586648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.587301 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.587302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.588090 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.588091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.588435 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.588435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.588543 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.589029 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:18.589030 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:18.589070 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:18.589094 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:18.589281 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:18.589286 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:18.589288 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.589342 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.589461 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260619 20:48:18.590222 1550 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40619
I20260619 20:48:18.590332 1550 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:18.590584 1550 heartbeater.cc:511] Master 127.13.195.126:40619 requested a full tablet report, sending...
I20260619 20:48:18.590971 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1421
I20260619 20:48:18.591099 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-0/wal/instance
I20260619 20:48:18.591327 14093 external_mini_cluster.cc:1428] Setting key 1b642f2ac853fbdc17a0ff3b697789aa
I20260619 20:48:18.591317 1361 ts_manager.cc:194] Registered new tserver with Master: b532160676c748b7a197f1aad5920161 (127.13.195.65:38783)
I20260619 20:48:18.592031 1361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.65:34363
I20260619 20:48:18.592600 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.66:0
--local_ip_for_outbound_sockets=127.13.195.66
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40619
--builtin_ntp_servers=127.13.195.84:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:48:18.677647 1553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:18.677839 1553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:18.677860 1553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:18.678988 1553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:18.679035 1553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:18.679060 1553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.66
I20260619 20:48:18.680454 1553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.195.66
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.66
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:18.680655 1553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:18.680895 1553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:18.685797 1559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.685928 1553 server_base.cc:1034] running on GCE node
W20260619 20:48:18.685823 1558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.685797 1561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.686331 1553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:18.686595 1553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:18.687742 1553 hybrid_clock.cc:648] HybridClock initialized: now 1781902098687728 us; error 29 us; skew 500 ppm
I20260619 20:48:18.689054 1553 webserver.cc:460] Webserver started at http://127.13.195.66:35623/ using document root <none> and password file <none>
I20260619 20:48:18.689289 1553 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:18.689338 1553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:18.689538 1553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:18.690672 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data/instance:
uuid: "454dc8ad263146f8819bbd9f0dc7401c"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "cf22e6b0804bd6a90ad95a138a1987fa"
server_key_iv: "66822b495d5241e200e8241bd2568c3f"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.691017 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance:
uuid: "454dc8ad263146f8819bbd9f0dc7401c"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "cf22e6b0804bd6a90ad95a138a1987fa"
server_key_iv: "66822b495d5241e200e8241bd2568c3f"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.692279 1553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:18.692978 1567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.693145 1553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:18.693243 1553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal
uuid: "454dc8ad263146f8819bbd9f0dc7401c"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "cf22e6b0804bd6a90ad95a138a1987fa"
server_key_iv: "66822b495d5241e200e8241bd2568c3f"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.693321 1553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:18.706846 1553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:18.707152 1553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:18.707302 1553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:18.707536 1553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:18.707861 1553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:18.707916 1553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.707957 1553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:18.707985 1553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.713781 1553 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.66:42209
I20260619 20:48:18.713829 1681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.66:42209 every 8 connection(s)
I20260619 20:48:18.714145 1553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/data/info.pb
I20260619 20:48:18.717767 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1553
I20260619 20:48:18.717882 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal/instance
I20260619 20:48:18.718101 14093 external_mini_cluster.cc:1428] Setting key e508cc9aaa61fc8320f37039a033add0
I20260619 20:48:18.719167 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.195.67:0
--local_ip_for_outbound_sockets=127.13.195.67
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40619
--builtin_ntp_servers=127.13.195.84:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 20:48:18.720037 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.714851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.66:33451 (local address 127.13.195.126:40619)
0619 20:48:18.714916 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.714918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.715151 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:48:18.715707 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.715708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.715730 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.715789 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.716507 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.717003 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.718333 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.718616 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.718681 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.719709 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.719710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.719710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:18.719716 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.719734 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:18.719739 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.719805 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.719882 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.719915 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:48:18.720160 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.714848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40619 (local address 127.13.195.66:33451)
0619 20:48:18.715030 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.715067 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.715597 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.715896 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.715900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.715908 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:18.716410 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.716415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.717122 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.717123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.718219 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.718223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.718701 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.718703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.718896 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.719557 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:18.719557 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:18.719587 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:18.719611 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:18.719807 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:18.719809 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:18.719811 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.719864 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.719954 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260619 20:48:18.720745 1682 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40619
I20260619 20:48:18.720858 1682 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:18.721076 1682 heartbeater.cc:511] Master 127.13.195.126:40619 requested a full tablet report, sending...
I20260619 20:48:18.721463 1361 ts_manager.cc:194] Registered new tserver with Master: 454dc8ad263146f8819bbd9f0dc7401c (127.13.195.66:42209)
I20260619 20:48:18.721849 1361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.66:33451
W20260619 20:48:18.801859 1685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:18.802109 1685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:18.802146 1685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:18.803376 1685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:18.803444 1685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:18.803491 1685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.195.67
I20260619 20:48:18.804963 1685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.195.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.195.67
--webserver_port=0
--tserver_master_addrs=127.13.195.126:40619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.195.67
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:18.805235 1685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:18.805537 1685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:18.810600 1691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.810678 1685 server_base.cc:1034] running on GCE node
W20260619 20:48:18.810647 1690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:18.810643 1693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:18.811012 1685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:18.811316 1685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:18.812485 1685 hybrid_clock.cc:648] HybridClock initialized: now 1781902098812465 us; error 37 us; skew 500 ppm
I20260619 20:48:18.813669 1685 webserver.cc:460] Webserver started at http://127.13.195.67:36573/ using document root <none> and password file <none>
I20260619 20:48:18.813925 1685 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:18.813995 1685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:18.814167 1685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:18.815285 1685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data/instance:
uuid: "3dc6c3968fce4f9ca5a678f996988e50"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "882ff1f4d31367f5ae68d389b714d30c"
server_key_iv: "8e5d4723d0819752420f7ea5cc8bfebb"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.815672 1685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance:
uuid: "3dc6c3968fce4f9ca5a678f996988e50"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "882ff1f4d31367f5ae68d389b714d30c"
server_key_iv: "8e5d4723d0819752420f7ea5cc8bfebb"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.816951 1685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:48:18.817714 1699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.817880 1685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:18.817976 1685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal
uuid: "3dc6c3968fce4f9ca5a678f996988e50"
format_stamp: "Formatted at 2026-06-19 20:48:18 on dist-test-slave-n2td"
server_key: "882ff1f4d31367f5ae68d389b714d30c"
server_key_iv: "8e5d4723d0819752420f7ea5cc8bfebb"
server_key_version: "encryptionkey@0"
I20260619 20:48:18.818077 1685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:18.832079 1685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:18.832392 1685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:18.832542 1685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:18.832851 1685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:48:18.833216 1685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:48:18.833277 1685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.833319 1685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:48:18.833349 1685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:18.839332 1685 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.67:37811
I20260619 20:48:18.839381 1813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.67:37811 every 8 connection(s)
I20260619 20:48:18.839704 1685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/data/info.pb
I20260619 20:48:18.844812 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.840451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:34645 (local address 127.13.195.126:40619)
0619 20:48:18.840515 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.840517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.840761 (+ 244us) server_negotiation.cc:407] Connection header received
0619 20:48:18.841238 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.841239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.841250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.841303 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.841974 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.842548 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.843502 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.843702 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.843752 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.844589 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.844590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.844590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:18.844594 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.844603 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:18.844606 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.844645 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.844730 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.844753 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:48:18.844959 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1685
I20260619 20:48:18.845077 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal/instance
I20260619 20:48:18.845091 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.840450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:40619 (local address 127.13.195.67:34645)
0619 20:48:18.840637 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.840671 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.841150 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.841385 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.841389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.841397 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:18.841900 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.841904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.842639 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.842640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.843425 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.843427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.843778 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.843779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.843884 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.844453 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:48:18.844454 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:18.844483 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:48:18.844505 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:18.844660 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:18.844664 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:18.844666 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.844719 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.844844 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260619 20:48:18.845335 14093 external_mini_cluster.cc:1428] Setting key a205dbdef9394ddf8442f9a39d3ef926
I20260619 20:48:18.845640 1814 heartbeater.cc:347] Connected to a master server at 127.13.195.126:40619
I20260619 20:48:18.845731 1814 heartbeater.cc:464] Registering TS with master...
I20260619 20:48:18.845923 1814 heartbeater.cc:511] Master 127.13.195.126:40619 requested a full tablet report, sending...
I20260619 20:48:18.846298 1361 ts_manager.cc:194] Registered new tserver with Master: 3dc6c3968fce4f9ca5a678f996988e50 (127.13.195.67:37811)
I20260619 20:48:18.846704 1361 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.195.67:34645
I20260619 20:48:18.857295 14093 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:48:18.861814 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.858740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51782 (local address 127.13.195.126:40619)
0619 20:48:18.858807 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.858810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.858927 (+ 117us) server_negotiation.cc:407] Connection header received
0619 20:48:18.859017 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.859018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.859031 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.859095 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:18.859300 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.859782 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.860556 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.860749 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.860823 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.861544 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:18.861546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:18.861546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:18.861551 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:18.861566 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:18.861572 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:18.861638 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.861698 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.861722 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:48:18.863525 1361 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:48:18.865016 1361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:48:18.871343 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.867093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38783 (local address 127.0.0.1:47750)
0619 20:48:18.867381 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.867425 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.867533 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.867833 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.867833 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.867835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.867888 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.867890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.868863 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.868864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.870113 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.870116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.871023 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.871024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.871113 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.871127 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.871213 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.871275 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":4,"thread_start_us":97,"threads_started":1}
I20260619 20:48:18.871361 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.867172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47750 (local address 127.13.195.65:38783)
0619 20:48:18.867347 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.867352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.867491 (+ 139us) server_negotiation.cc:407] Connection header received
0619 20:48:18.867650 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.867653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.867670 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.867727 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.868041 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.868770 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.870196 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.870928 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.871019 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.871118 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.871230 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.871273 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260619 20:48:18.871554 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.867098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:42209 (local address 127.0.0.1:53658)
0619 20:48:18.867486 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.867497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.867551 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.867803 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.867806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.867808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.867888 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.867892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.869234 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.869235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.870297 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.870299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.871267 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.871268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.871336 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.871340 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.871391 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.871436 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260619 20:48:18.871788 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.867189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.13.195.66:42209)
0619 20:48:18.867471 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.867475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.867634 (+ 159us) server_negotiation.cc:407] Connection header received
0619 20:48:18.867682 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.867684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.867701 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.867987 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.868067 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.869145 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.870373 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.871092 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.871155 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.871233 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.871661 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.871706 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":74,"threads_started":1}
I20260619 20:48:18.872193 1484 tablet_service.cc:1467] Processing CreateTablet for tablet ab65f5e6448e46c68facf2521b082290 (DEFAULT_TABLE table=test-table [id=e7aea62a62114d0d948157b7dd90ecfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:18.872431 1616 tablet_service.cc:1467] Processing CreateTablet for tablet ab65f5e6448e46c68facf2521b082290 (DEFAULT_TABLE table=test-table [id=e7aea62a62114d0d948157b7dd90ecfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:18.872572 1484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab65f5e6448e46c68facf2521b082290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:18.872617 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.867364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.67:37811 (local address 127.0.0.1:53328)
0619 20:48:18.867524 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:18.867572 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:18.867618 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:18.867869 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:18.867870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:18.867872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:18.867919 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.867921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.870365 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.870366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.871181 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:18.871183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.872387 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.872388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:18.872449 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.872452 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:18.872512 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:48:18.872550 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
I20260619 20:48:18.872634 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:18.867447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.13.195.67:37811)
0619 20:48:18.867673 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:18.867677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:18.867685 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:18.867720 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:18.867723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:18.867738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:18.867806 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:18.867997 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.869150 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.871489 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:18.872284 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:18.872383 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:18.872467 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:18.872526 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 20:48:18.872559 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":117,"threads_started":1}
I20260619 20:48:18.872746 1616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab65f5e6448e46c68facf2521b082290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:18.873246 1747 tablet_service.cc:1467] Processing CreateTablet for tablet ab65f5e6448e46c68facf2521b082290 (DEFAULT_TABLE table=test-table [id=e7aea62a62114d0d948157b7dd90ecfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:48:18.873474 1747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab65f5e6448e46c68facf2521b082290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:18.875818 1828 tablet_bootstrap.cc:492] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Bootstrap starting.
I20260619 20:48:18.876372 1829 tablet_bootstrap.cc:492] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161: Bootstrap starting.
I20260619 20:48:18.876508 1828 tablet_bootstrap.cc:654] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:18.876783 1828 log.cc:824] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:18.876963 1829 tablet_bootstrap.cc:654] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:18.877328 1829 log.cc:824] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:18.877578 1828 tablet_bootstrap.cc:492] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: No bootstrap required, opened a new log
I20260619 20:48:18.877656 1828 ts_tablet_manager.cc:1397] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:48:18.877650 1830 tablet_bootstrap.cc:492] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Bootstrap starting.
I20260619 20:48:18.878163 1830 tablet_bootstrap.cc:654] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:18.878501 1830 log.cc:824] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:18.878809 1829 tablet_bootstrap.cc:492] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161: No bootstrap required, opened a new log
I20260619 20:48:18.878899 1829 ts_tablet_manager.cc:1397] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:48:18.879696 1830 tablet_bootstrap.cc:492] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: No bootstrap required, opened a new log
I20260619 20:48:18.879810 1830 ts_tablet_manager.cc:1397] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:48:18.880249 1828 raft_consensus.cc:348] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:18.880453 1828 raft_consensus.cc:374] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:18.880545 1828 raft_consensus.cc:729] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dc6c3968fce4f9ca5a678f996988e50, State: Initialized, Role: FOLLOWER
I20260619 20:48:18.880677 1828 consensus_queue.cc:260] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:18.880957 1828 ts_tablet_manager.cc:1428] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 20:48:18.881139 1814 heartbeater.cc:503] Master 127.13.195.126:40619 was elected leader, sending a full tablet report...
I20260619 20:48:18.881517 1829 raft_consensus.cc:348] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:18.881708 1829 raft_consensus.cc:374] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:18.881743 1829 raft_consensus.cc:729] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b532160676c748b7a197f1aad5920161, State: Initialized, Role: FOLLOWER
I20260619 20:48:18.881913 1829 consensus_queue.cc:260] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:18.882171 1829 ts_tablet_manager.cc:1428] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 20:48:18.882216 1550 heartbeater.cc:503] Master 127.13.195.126:40619 was elected leader, sending a full tablet report...
I20260619 20:48:18.882503 1830 raft_consensus.cc:348] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:18.882676 1830 raft_consensus.cc:374] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:18.882731 1830 raft_consensus.cc:729] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454dc8ad263146f8819bbd9f0dc7401c, State: Initialized, Role: FOLLOWER
I20260619 20:48:18.882848 1830 consensus_queue.cc:260] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:18.883103 1830 ts_tablet_manager.cc:1428] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 20:48:18.883131 1682 heartbeater.cc:503] Master 127.13.195.126:40619 was elected leader, sending a full tablet report...
W20260619 20:48:18.964687 1683 tablet.cc:2369] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:48:19.042502 1834 raft_consensus.cc:482] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:48:19.042671 1834 raft_consensus.cc:504] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:19.042991 1834 leader_election.cc:288] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 454dc8ad263146f8819bbd9f0dc7401c (127.13.195.66:42209), b532160676c748b7a197f1aad5920161 (127.13.195.65:38783)
I20260619 20:48:19.046031 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:19.043212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.66:42209 (local address 127.13.195.67:46537)
0619 20:48:19.043260 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:19.043272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:19.043319 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:19.043600 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:19.043602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:19.043604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:19.043685 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:19.043688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.044392 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.044393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:19.045198 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:19.045201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.045766 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.045767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:19.045847 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:19.045851 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:48:19.045904 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:48:19.045947 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 20:48:19.046067 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:19.043307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:46537 (local address 127.13.195.66:42209)
0619 20:48:19.043349 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:19.043351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:19.043372 (+ 21us) server_negotiation.cc:407] Connection header received
0619 20:48:19.043406 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:19.043407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:19.043418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:19.043506 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:19.043774 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.044307 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.045298 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.045683 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.045739 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:19.045745 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:19.045924 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 20:48:19.045965 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:48:19.046145 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:19.043213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38783 (local address 127.13.195.67:49481)
0619 20:48:19.043438 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:19.043448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:19.043512 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:19.043750 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:19.043751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:19.043753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:48:19.043827 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:19.043829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.044499 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.044500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:19.045278 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:19.045281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.045866 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.045867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:19.045961 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:19.045964 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:48:19.046023 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:48:19.046059 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":55,"thread_start_us":103,"threads_started":1}
I20260619 20:48:19.046170 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:19.043297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.195.67:49481 (local address 127.13.195.65:38783)
0619 20:48:19.043347 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:19.043350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:19.043579 (+ 229us) server_negotiation.cc:407] Connection header received
0619 20:48:19.043625 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:19.043626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:19.043639 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:19.043712 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:48:19.043908 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.044415 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.045368 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.045781 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.045852 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:19.045860 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:19.046048 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 20:48:19.046097 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:48:19.046582 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab65f5e6448e46c68facf2521b082290" candidate_uuid: "3dc6c3968fce4f9ca5a678f996988e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "454dc8ad263146f8819bbd9f0dc7401c" is_pre_election: true
I20260619 20:48:19.046566 1504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab65f5e6448e46c68facf2521b082290" candidate_uuid: "3dc6c3968fce4f9ca5a678f996988e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b532160676c748b7a197f1aad5920161" is_pre_election: true
I20260619 20:48:19.046749 1504 raft_consensus.cc:2454] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dc6c3968fce4f9ca5a678f996988e50 in term 0.
I20260619 20:48:19.046749 1636 raft_consensus.cc:2454] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dc6c3968fce4f9ca5a678f996988e50 in term 0.
I20260619 20:48:19.047010 1703 leader_election.cc:302] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dc6c3968fce4f9ca5a678f996988e50, b532160676c748b7a197f1aad5920161; no voters:
I20260619 20:48:19.047163 1834 raft_consensus.cc:2781] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:48:19.047228 1834 raft_consensus.cc:482] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:48:19.047245 1834 raft_consensus.cc:3037] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:19.048050 1834 raft_consensus.cc:504] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:19.048228 1834 leader_election.cc:288] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [CANDIDATE]: Term 1 election: Requested vote from peers 454dc8ad263146f8819bbd9f0dc7401c (127.13.195.66:42209), b532160676c748b7a197f1aad5920161 (127.13.195.65:38783)
I20260619 20:48:19.048413 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab65f5e6448e46c68facf2521b082290" candidate_uuid: "3dc6c3968fce4f9ca5a678f996988e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "454dc8ad263146f8819bbd9f0dc7401c"
I20260619 20:48:19.048408 1504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab65f5e6448e46c68facf2521b082290" candidate_uuid: "3dc6c3968fce4f9ca5a678f996988e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b532160676c748b7a197f1aad5920161"
I20260619 20:48:19.048512 1636 raft_consensus.cc:3037] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:19.048509 1504 raft_consensus.cc:3037] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:19.049301 1636 raft_consensus.cc:2454] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dc6c3968fce4f9ca5a678f996988e50 in term 1.
I20260619 20:48:19.049326 1504 raft_consensus.cc:2454] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dc6c3968fce4f9ca5a678f996988e50 in term 1.
I20260619 20:48:19.049520 1703 leader_election.cc:302] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dc6c3968fce4f9ca5a678f996988e50, b532160676c748b7a197f1aad5920161; no voters:
I20260619 20:48:19.049631 1834 raft_consensus.cc:2781] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:19.049801 1834 raft_consensus.cc:686] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 1 LEADER]: Becoming Leader. State: Replica: 3dc6c3968fce4f9ca5a678f996988e50, State: Running, Role: LEADER
I20260619 20:48:19.049919 1834 consensus_queue.cc:237] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } }
I20260619 20:48:19.050902 1360 catalog_manager.cc:5495] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dc6c3968fce4f9ca5a678f996988e50 (127.13.195.67). New cstate: current_term: 1 leader_uuid: "3dc6c3968fce4f9ca5a678f996988e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dc6c3968fce4f9ca5a678f996988e50" member_type: VOTER last_known_addr { host: "127.13.195.67" port: 37811 } health_report { overall_health: HEALTHY } } }
I20260619 20:48:19.056342 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:19.053897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.13.195.67:37811)
0619 20:48:19.053941 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:19.053943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:19.053951 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:19.053985 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:19.053986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:19.054003 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:19.054097 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:19.054241 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.054786 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.055566 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:19.055770 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:19.055846 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:19.056003 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:19.056115 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:19.056211 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:19.056230 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 20:48:19.056249 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:19.058135 1636 raft_consensus.cc:1261] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 1 FOLLOWER]: Refusing update from remote peer 3dc6c3968fce4f9ca5a678f996988e50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:19.058135 1504 raft_consensus.cc:1261] T ab65f5e6448e46c68facf2521b082290 P b532160676c748b7a197f1aad5920161 [term 1 FOLLOWER]: Refusing update from remote peer 3dc6c3968fce4f9ca5a678f996988e50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:19.058494 1838 consensus_queue.cc:1035] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b532160676c748b7a197f1aad5920161" member_type: VOTER last_known_addr { host: "127.13.195.65" port: 38783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:19.058669 1834 consensus_queue.cc:1035] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "454dc8ad263146f8819bbd9f0dc7401c" member_type: VOTER last_known_addr { host: "127.13.195.66" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:19.060993 1841 mvcc.cc:205] Tried to move back new op lower bound from 7298670997739335680 to 7298670997709402112. Current Snapshot: MvccSnapshot[applied={T|T < 7298670997739335680}]
I20260619 20:48:19.061606 1843 mvcc.cc:205] Tried to move back new op lower bound from 7298670997739335680 to 7298670997709402112. Current Snapshot: MvccSnapshot[applied={T|T < 7298670997739335680 or (T in {7298670997739335680})}]
I20260619 20:48:19.063447 1360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:19.063539 1360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:48:19.064296 1360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cb96d3bf9284d48808b03af61ca6621: Sending DeleteTablet for 3 replicas of tablet ab65f5e6448e46c68facf2521b082290
I20260619 20:48:19.064662 1484 tablet_service.cc:1514] Processing DeleteTablet for tablet ab65f5e6448e46c68facf2521b082290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:19 UTC) from {username='slave'} at 127.0.0.1:47750
I20260619 20:48:19.064664 1616 tablet_service.cc:1514] Processing DeleteTablet for tablet ab65f5e6448e46c68facf2521b082290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:19 UTC) from {username='slave'} at 127.0.0.1:53658
I20260619 20:48:19.064885 1851 tablet_replica.cc:330] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: stopping tablet replica
I20260619 20:48:19.064915 1747 tablet_service.cc:1514] Processing DeleteTablet for tablet ab65f5e6448e46c68facf2521b082290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:48:19 UTC) from {username='slave'} at 127.0.0.1:53328
I20260619 20:48:19.065014 1851 raft_consensus.cc:2229] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:48:19.065110 1851 raft_consensus.cc:2258] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:19.065165 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1421
I20260619 20:48:19.065958 1852 tablet_replica.cc:330] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: stopping tablet replica
I20260619 20:48:19.066059 1852 raft_consensus.cc:2229] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:48:19.066147 1852 raft_consensus.cc:2258] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:48:19.066656 1852 ts_tablet_manager.cc:1905] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:19.067307 1851 ts_tablet_manager.cc:1905] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:48:19.068699 1852 ts_tablet_manager.cc:1918] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:19.068838 1852 log.cc:1196] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-2/wal/wals/ab65f5e6448e46c68facf2521b082290
I20260619 20:48:19.068994 1851 ts_tablet_manager.cc:1918] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:48:19.069152 1851 log.cc:1196] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Deleting WAL directory at /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901857247738-14093-0/minicluster-data/ts-1/wal/wals/ab65f5e6448e46c68facf2521b082290
I20260619 20:48:19.069170 1852 ts_tablet_manager.cc:1939] T ab65f5e6448e46c68facf2521b082290 P 3dc6c3968fce4f9ca5a678f996988e50: Deleting consensus metadata
I20260619 20:48:19.069487 1851 ts_tablet_manager.cc:1939] T ab65f5e6448e46c68facf2521b082290 P 454dc8ad263146f8819bbd9f0dc7401c: Deleting consensus metadata
I20260619 20:48:19.069702 1345 catalog_manager.cc:4841] TS 3dc6c3968fce4f9ca5a678f996988e50 (127.13.195.67:37811): tablet ab65f5e6448e46c68facf2521b082290 (table test-table [id=e7aea62a62114d0d948157b7dd90ecfd]) successfully deleted
I20260619 20:48:19.069940 1345 catalog_manager.cc:4841] TS 454dc8ad263146f8819bbd9f0dc7401c (127.13.195.66:42209): tablet ab65f5e6448e46c68facf2521b082290 (table test-table [id=e7aea62a62114d0d948157b7dd90ecfd]) successfully deleted
W20260619 20:48:19.071332 1348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.195.65:38783 (error 108)
I20260619 20:48:19.071539 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1553
I20260619 20:48:19.072652 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:19.072185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.65:38783 (local address 127.0.0.1:47752)
0619 20:48:19.072557 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:19.072610 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345}
W20260619 20:48:19.072815 1348 catalog_manager.cc:4584] TS b532160676c748b7a197f1aad5920161 (127.13.195.65:38783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab65f5e6448e46c68facf2521b082290: Network error: Client connection negotiation failed: client connection to 127.13.195.65:38783: connect: Connection refused (error 111)
I20260619 20:48:19.077261 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1685
I20260619 20:48:19.082127 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1330
2026-06-19T20:48:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (801 ms)
[----------] 6 tests from EncryptionPolicyTest (8094 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:19 dist-test-slave-n2td krb5kdc[1856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:19 dist-test-slave-n2td krb5kdc[1856](info): set up 2 sockets
Jun 19 20:48:19 dist-test-slave-n2td krb5kdc[1856](info): commencing operation
krb5kdc: starting...
W20260619 20:48:20.129283 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:20 dist-test-slave-n2td krb5kdc[1856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:20.245507 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:20.328053 1869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:20.328293 1869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:20.328318 1869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:20.329499 1869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:20.329543 1869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:20.329557 1869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:20.329571 1869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:20.329627 1869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:20.331243 1869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:20.331727 1869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:20.332029 1869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:20.337908 1875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:20.337895 1876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:20.337908 1878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:20.338307 1869 server_base.cc:1034] running on GCE node
I20260619 20:48:20.338629 1869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:20.338943 1869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:20.340129 1869 hybrid_clock.cc:648] HybridClock initialized: now 1781902100340130 us; error 82 us; skew 500 ppm
Jun 19 20:48:20 dist-test-slave-n2td krb5kdc[1856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:20.342702 1869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:20.343092 1869 webserver.cc:460] Webserver started at http://127.0.0.1:45187/ using document root <none> and password file <none>
I20260619 20:48:20.343344 1869 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:20.343390 1869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:20.343542 1869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:20.344698 1869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "143bec9947464a7693f94ba2bd58a4b5"
format_stamp: "Formatted at 2026-06-19 20:48:20 on dist-test-slave-n2td"
server_key: "e4426881d5ae222f2e7f669002a2a949"
server_key_iv: "da4d1eb5f4663db30f7899daffed47f6"
server_key_version: "encryptionkey@0"
I20260619 20:48:20.345064 1869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "143bec9947464a7693f94ba2bd58a4b5"
format_stamp: "Formatted at 2026-06-19 20:48:20 on dist-test-slave-n2td"
server_key: "e4426881d5ae222f2e7f669002a2a949"
server_key_iv: "da4d1eb5f4663db30f7899daffed47f6"
server_key_version: "encryptionkey@0"
I20260619 20:48:20.346361 1869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:20.347146 1885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:20.347332 1869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:20.347396 1869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "143bec9947464a7693f94ba2bd58a4b5"
format_stamp: "Formatted at 2026-06-19 20:48:20 on dist-test-slave-n2td"
server_key: "e4426881d5ae222f2e7f669002a2a949"
server_key_iv: "da4d1eb5f4663db30f7899daffed47f6"
server_key_version: "encryptionkey@0"
I20260619 20:48:20.347489 1869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:20.356582 1869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:20.358350 1869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:20.358593 1869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:20.362421 1869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32981
I20260619 20:48:20.362519 1937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32981 every 8 connection(s)
I20260619 20:48:20.362864 1869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:20.363710 1938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:20.365921 1938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5: Bootstrap starting.
I20260619 20:48:20.366565 1938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:20.366863 1938 log.cc:824] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:20.367539 1938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5: No bootstrap required, opened a new log
I20260619 20:48:20.369124 1938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143bec9947464a7693f94ba2bd58a4b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32981 } }
I20260619 20:48:20.369259 1938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:20.369282 1938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 143bec9947464a7693f94ba2bd58a4b5, State: Initialized, Role: FOLLOWER
I20260619 20:48:20.369467 1938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143bec9947464a7693f94ba2bd58a4b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32981 } }
I20260619 20:48:20.369551 1938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:20.369575 1938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:20.369621 1938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:20.370132 1938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143bec9947464a7693f94ba2bd58a4b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32981 } }
I20260619 20:48:20.370213 1938 leader_election.cc:302] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 143bec9947464a7693f94ba2bd58a4b5; no voters:
I20260619 20:48:20.370410 1938 leader_election.cc:288] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:20.370455 1941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:20.370574 1941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [term 1 LEADER]: Becoming Leader. State: Replica: 143bec9947464a7693f94ba2bd58a4b5, State: Running, Role: LEADER
I20260619 20:48:20.370599 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1869
I20260619 20:48:20.370728 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:20.370707 1938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:20.370714 1941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143bec9947464a7693f94ba2bd58a4b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32981 } }
I20260619 20:48:20.370978 14093 external_mini_cluster.cc:1428] Setting key ce6842abff84080504554cba28888363
I20260619 20:48:20.371055 1941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 143bec9947464a7693f94ba2bd58a4b5. Latest consensus state: current_term: 1 leader_uuid: "143bec9947464a7693f94ba2bd58a4b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143bec9947464a7693f94ba2bd58a4b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32981 } } }
I20260619 20:48:20.371127 1941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:20.371227 1942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "143bec9947464a7693f94ba2bd58a4b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143bec9947464a7693f94ba2bd58a4b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32981 } } }
I20260619 20:48:20.371308 1942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:20.372089 1947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:20.372627 1947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:20.374137 1947 catalog_manager.cc:1347] Generated new cluster ID: 787c4147fe724168a9097929a9f0b500
I20260619 20:48:20.374190 1947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:20 dist-test-slave-n2td krb5kdc[1856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:48:20.381692 1953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:20.371637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42321 (local address 127.0.0.1:32981)
0619 20:48:20.372101 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:20.372111 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:20.372144 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:48:20.372262 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:20.372269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:20.372329 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:20.372531 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:20.373174 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:20.374495 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:20.375236 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:20.375661 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:20.375773 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:20.378106 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:20.378114 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:20.378114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:20.378125 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:20.380074 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:20.380476 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:20.380480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:20.380482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:20.380602 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:20.380834 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:20.380835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:20.380836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:20.381101 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:20.381288 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:20.381375 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:48:20.381478 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":97,"threads_started":1}
I20260619 20:48:20.382706 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:20.392807 1953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:20.384185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48587 (local address 127.0.0.1:32981)
0619 20:48:20.384226 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:20.384228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:20.386657 (+ 2429us) server_negotiation.cc:407] Connection header received
0619 20:48:20.386735 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:20.386736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:20.386749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:20.386798 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:20.387362 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:20.387867 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:20.388582 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:20.388814 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:20.388897 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:20.390119 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:20.390120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:20.390120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:20.390128 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:20.391592 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:20.391976 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:20.391977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:20.391977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:20.392038 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:20.392267 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:20.392268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:20.392268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:20.392383 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:20.392510 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:20.392628 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 20:48:20.392689 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:20.393970 1947 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:20.394646 1947 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:20.401350 1947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 143bec9947464a7693f94ba2bd58a4b5: Generated new TSK 0
I20260619 20:48:20.401538 1947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:20.409219 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1869
W20260619 20:48:20.413534 1854 connection.cc:535] client connection to 127.0.0.1:32981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32981): Connection reset by peer (error 104)
2026-06-19T20:48:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:20 dist-test-slave-n2td krb5kdc[1970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:20 dist-test-slave-n2td krb5kdc[1970](info): set up 2 sockets
Jun 19 20:48:20 dist-test-slave-n2td krb5kdc[1970](info): commencing operation
krb5kdc: starting...
W20260619 20:48:21.437230 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:21 dist-test-slave-n2td krb5kdc[1970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:21.557508 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:21.646322 1983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:21.646610 1983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:21.646653 1983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:21.647814 1983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:21.647887 1983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:21.647912 1983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:21.647939 1983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:21.647964 1983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:21.649428 1983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45655
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:21.649710 1983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:21.649940 1983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:21.654800 1989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:21.654984 1983 server_base.cc:1034] running on GCE node
W20260619 20:48:21.654801 1991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:21.655124 1988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:21.655375 1983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:21.655679 1983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:21.656909 1983 hybrid_clock.cc:648] HybridClock initialized: now 1781902101656889 us; error 33 us; skew 500 ppm
Jun 19 20:48:21 dist-test-slave-n2td krb5kdc[1970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:21.662985 1983 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:21.663453 1983 webserver.cc:460] Webserver started at http://127.0.0.1:45651/ using document root <none> and password file <none>
I20260619 20:48:21.663698 1983 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:21.663771 1983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:21.663933 1983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:21.665058 1983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "2040f17d60ac4cd69f9a8c64dafc9646"
format_stamp: "Formatted at 2026-06-19 20:48:21 on dist-test-slave-n2td"
server_key: "599499eed36bc543e84a5077b6be60ed"
server_key_iv: "273998f4438a55b4109e8154c93d73fb"
server_key_version: "encryptionkey@0"
I20260619 20:48:21.665423 1983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "2040f17d60ac4cd69f9a8c64dafc9646"
format_stamp: "Formatted at 2026-06-19 20:48:21 on dist-test-slave-n2td"
server_key: "599499eed36bc543e84a5077b6be60ed"
server_key_iv: "273998f4438a55b4109e8154c93d73fb"
server_key_version: "encryptionkey@0"
I20260619 20:48:21.666852 1983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:48:21.667637 1998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:21.667806 1983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:21.667912 1983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "2040f17d60ac4cd69f9a8c64dafc9646"
format_stamp: "Formatted at 2026-06-19 20:48:21 on dist-test-slave-n2td"
server_key: "599499eed36bc543e84a5077b6be60ed"
server_key_iv: "273998f4438a55b4109e8154c93d73fb"
server_key_version: "encryptionkey@0"
I20260619 20:48:21.667987 1983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:21.682545 1983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:21.684942 1983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:21.685118 1983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:21.689321 1983 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45655
I20260619 20:48:21.689363 2050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45655 every 8 connection(s)
I20260619 20:48:21.689739 1983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:21.690750 2051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:21.692598 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 1983
I20260619 20:48:21.692755 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:21.692965 14093 external_mini_cluster.cc:1428] Setting key 73beb3c4f941ef69c2607a5d9c944ac7
I20260619 20:48:21.693634 2051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646: Bootstrap starting.
I20260619 20:48:21.694304 2051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:21.694679 2051 log.cc:824] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:21.695453 2051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646: No bootstrap required, opened a new log
I20260619 20:48:21.697400 2051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260619 20:48:21.697580 2051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:21.697633 2051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2040f17d60ac4cd69f9a8c64dafc9646, State: Initialized, Role: FOLLOWER
I20260619 20:48:21.697796 2051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260619 20:48:21.697902 2051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:21.697952 2051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:21.698005 2051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:21.698479 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:21.693559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48135 (local address 127.0.0.1:45655)
0619 20:48:21.693731 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:21.693739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:21.693786 (+ 47us) server_negotiation.cc:407] Connection header received
0619 20:48:21.694229 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:21.694236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:21.694283 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:21.694442 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:21.694651 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:21.695639 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:21.696474 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:21.696888 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:21.697014 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:21.697764 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:21.697768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:21.697769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:21.697978 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:21.698024 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:21.698037 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:21.698117 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:21.698197 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:48:21.698226 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260619 20:48:21.698700 2051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260619 20:48:21.698829 2051 leader_election.cc:302] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2040f17d60ac4cd69f9a8c64dafc9646; no voters:
I20260619 20:48:21.699030 2051 leader_election.cc:288] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:21.699198 2057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:21.699304 2051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:21.699379 2057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [term 1 LEADER]: Becoming Leader. State: Replica: 2040f17d60ac4cd69f9a8c64dafc9646, State: Running, Role: LEADER
I20260619 20:48:21.699496 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:21.699491 2057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } }
I20260619 20:48:21.699828 2057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2040f17d60ac4cd69f9a8c64dafc9646. Latest consensus state: current_term: 1 leader_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } } }
I20260619 20:48:21.699947 2057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:21.700191 2064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:21.700881 2064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:21.701233 2056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2040f17d60ac4cd69f9a8c64dafc9646" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45655 } } }
I20260619 20:48:21.701515 2056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:21.702392 2064 catalog_manager.cc:1347] Generated new cluster ID: 0af25f3113894cf98c52551fcb819d11
I20260619 20:48:21.702458 2064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:21.704963 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:21.701454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33915 (local address 127.0.0.1:45655)
0619 20:48:21.701494 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:21.701497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:21.701748 (+ 251us) server_negotiation.cc:407] Connection header received
0619 20:48:21.701832 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:21.701833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:21.701844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:21.701927 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:21.702083 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:21.702796 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:21.703602 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:21.703835 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:21.703917 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:21.704661 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:21.704662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:21.704662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:21.704668 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:21.704683 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:21.704687 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:21.704772 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:21.704843 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 20:48:21.704853 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:21.707405 2064 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:21.708155 2064 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:21.714984 2064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2040f17d60ac4cd69f9a8c64dafc9646: Generated new TSK 0
I20260619 20:48:21.715180 2064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:21.718904 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 1983
W20260619 20:48:21.723685 1968 connection.cc:535] client connection to 127.0.0.1:45655 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45655): Connection reset by peer (error 104)
2026-06-19T20:48:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:21 dist-test-slave-n2td krb5kdc[2082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:21 dist-test-slave-n2td krb5kdc[2082](info): set up 2 sockets
Jun 19 20:48:21 dist-test-slave-n2td krb5kdc[2082](info): commencing operation
krb5kdc: starting...
W20260619 20:48:22.773274 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:22 dist-test-slave-n2td krb5kdc[2082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:22.889626 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:22.969403 2095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:22.969615 2095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:22.969637 2095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:22.970729 2095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:22.970765 2095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:22.970779 2095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:22.970793 2095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:22.970804 2095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:22.972227 2095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:22.972417 2095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:22.972584 2095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:22.977355 2103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:22.977346 2100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:22.977342 2101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:22.977584 2095 server_base.cc:1034] running on GCE node
I20260619 20:48:22.977866 2095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:22.978163 2095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:22.979354 2095 hybrid_clock.cc:648] HybridClock initialized: now 1781902102979336 us; error 53 us; skew 500 ppm
Jun 19 20:48:22 dist-test-slave-n2td krb5kdc[2082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902102, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:22.981866 2095 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:22.982343 2095 webserver.cc:460] Webserver started at http://127.0.0.1:35511/ using document root <none> and password file <none>
I20260619 20:48:22.982651 2095 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:22.982729 2095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:22.982899 2095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:22.984079 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "b90e323a4cf34853b0ede3208762853b"
format_stamp: "Formatted at 2026-06-19 20:48:22 on dist-test-slave-n2td"
server_key: "6bd1b984acc86c16cfb44e8af47c72fe"
server_key_iv: "816e4c22ede54a21dc4be99cbfcfbb66"
server_key_version: "encryptionkey@0"
I20260619 20:48:22.984483 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "b90e323a4cf34853b0ede3208762853b"
format_stamp: "Formatted at 2026-06-19 20:48:22 on dist-test-slave-n2td"
server_key: "6bd1b984acc86c16cfb44e8af47c72fe"
server_key_iv: "816e4c22ede54a21dc4be99cbfcfbb66"
server_key_version: "encryptionkey@0"
I20260619 20:48:22.985891 2095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:22.986727 2110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:22.986907 2095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:22.987037 2095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "b90e323a4cf34853b0ede3208762853b"
format_stamp: "Formatted at 2026-06-19 20:48:22 on dist-test-slave-n2td"
server_key: "6bd1b984acc86c16cfb44e8af47c72fe"
server_key_iv: "816e4c22ede54a21dc4be99cbfcfbb66"
server_key_version: "encryptionkey@0"
I20260619 20:48:22.987120 2095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:22.991074 2095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:22.992935 2095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:22.993098 2095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:22.996961 2095 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40207
I20260619 20:48:22.997013 2163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40207 every 8 connection(s)
I20260619 20:48:22.997365 2095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:22.998189 2164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:23.000420 2164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b: Bootstrap starting.
I20260619 20:48:23.000986 2164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:23.001271 2164 log.cc:824] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:23.001937 2164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b: No bootstrap required, opened a new log
I20260619 20:48:23.003640 2164 raft_consensus.cc:348] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90e323a4cf34853b0ede3208762853b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260619 20:48:23.003772 2164 raft_consensus.cc:374] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:23.003793 2164 raft_consensus.cc:729] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90e323a4cf34853b0ede3208762853b, State: Initialized, Role: FOLLOWER
I20260619 20:48:23.003964 2164 consensus_queue.cc:260] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90e323a4cf34853b0ede3208762853b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260619 20:48:23.004050 2164 raft_consensus.cc:388] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:23.004120 2164 raft_consensus.cc:482] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:23.004173 2164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:23.004323 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2095
I20260619 20:48:23.004462 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:23.004660 14093 external_mini_cluster.cc:1428] Setting key 41fb93ae86e2463ce59e64a0de5658d4
I20260619 20:48:23.004842 2164 raft_consensus.cc:504] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90e323a4cf34853b0ede3208762853b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260619 20:48:23.004954 2164 leader_election.cc:302] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b90e323a4cf34853b0ede3208762853b; no voters:
I20260619 20:48:23.005113 2164 leader_election.cc:288] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:23.005278 2169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:23.005390 2164 sys_catalog.cc:564] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:23.005430 2169 raft_consensus.cc:686] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [term 1 LEADER]: Becoming Leader. State: Replica: b90e323a4cf34853b0ede3208762853b, State: Running, Role: LEADER
I20260619 20:48:23.005514 2169 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90e323a4cf34853b0ede3208762853b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260619 20:48:23.006110 2167 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b90e323a4cf34853b0ede3208762853b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90e323a4cf34853b0ede3208762853b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260619 20:48:23.006201 2167 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:23.006902 2171 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90e323a4cf34853b0ede3208762853b. Latest consensus state: current_term: 1 leader_uuid: "b90e323a4cf34853b0ede3208762853b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90e323a4cf34853b0ede3208762853b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260619 20:48:23.007148 2171 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b [sys.catalog]: This master's current role is: LEADER
W20260619 20:48:23.007303 2184 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:23.007362 2184 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:23.007457 2185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:23.007658 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:23.005177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44943 (local address 127.0.0.1:40207)
0619 20:48:23.005666 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:23.005678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:23.005711 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:48:23.005827 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:23.005833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:23.005875 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:23.006287 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:23.007027 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:23.007032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:23.007032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:23.007037 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:23.007080 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:23.007095 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:23.007323 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:23.007360 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 20:48:23.007402 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":86,"threads_started":1}
I20260619 20:48:23.007658 2185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:23.008674 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:23.009163 2185 catalog_manager.cc:1347] Generated new cluster ID: e8fac2962120468baeb34f50317e92a1
I20260619 20:48:23.009203 2185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:23.011373 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:23.010294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44561 (local address 127.0.0.1:40207)
0619 20:48:23.010329 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:23.010331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:23.010423 (+ 92us) server_negotiation.cc:407] Connection header received
0619 20:48:23.010520 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:23.010521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:23.010525 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:23.010570 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:23.011168 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:23.011169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:23.011170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:23.011173 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:23.011187 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:23.011190 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:23.011242 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:23.011307 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 20:48:23.011318 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:23.023497 2185 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:23.023676 2185 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:23.029311 2185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b90e323a4cf34853b0ede3208762853b: Generated new TSK 0
I20260619 20:48:23.029541 2185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:23.032930 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2095
2026-06-19T20:48:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:23 dist-test-slave-n2td krb5kdc[2195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:23 dist-test-slave-n2td krb5kdc[2195](info): set up 2 sockets
Jun 19 20:48:23 dist-test-slave-n2td krb5kdc[2195](info): commencing operation
krb5kdc: starting...
W20260619 20:48:24.077494 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:24 dist-test-slave-n2td krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:24.196830 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:24.280018 2208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:24.280236 2208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:24.280259 2208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:24.281417 2208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:24.281461 2208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:24.281476 2208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:24.281488 2208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:24.281502 2208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:24.283015 2208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:24.283286 2208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:24.283536 2208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:24.288602 2213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:24.288582 2216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:24.288581 2214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:24.288645 2208 server_base.cc:1034] running on GCE node
I20260619 20:48:24.289005 2208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:24.289260 2208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:24.290400 2208 hybrid_clock.cc:648] HybridClock initialized: now 1781902104290379 us; error 39 us; skew 500 ppm
Jun 19 20:48:24 dist-test-slave-n2td krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:24.292953 2208 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:24.293390 2208 webserver.cc:460] Webserver started at http://127.0.0.1:36141/ using document root <none> and password file <none>
I20260619 20:48:24.293642 2208 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:24.293715 2208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:24.293867 2208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:24.295102 2208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "db70280a31a248f5a9b6ee5adcc65eeb"
format_stamp: "Formatted at 2026-06-19 20:48:24 on dist-test-slave-n2td"
server_key: "a8e992959de8455d00308c7f48dd3766"
server_key_iv: "e2d12884ad5e8ab06344c2352af4fc31"
server_key_version: "encryptionkey@0"
I20260619 20:48:24.295547 2208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "db70280a31a248f5a9b6ee5adcc65eeb"
format_stamp: "Formatted at 2026-06-19 20:48:24 on dist-test-slave-n2td"
server_key: "a8e992959de8455d00308c7f48dd3766"
server_key_iv: "e2d12884ad5e8ab06344c2352af4fc31"
server_key_version: "encryptionkey@0"
I20260619 20:48:24.297226 2208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:24.298403 2223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:24.298638 2208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:24.298756 2208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "db70280a31a248f5a9b6ee5adcc65eeb"
format_stamp: "Formatted at 2026-06-19 20:48:24 on dist-test-slave-n2td"
server_key: "a8e992959de8455d00308c7f48dd3766"
server_key_iv: "e2d12884ad5e8ab06344c2352af4fc31"
server_key_version: "encryptionkey@0"
I20260619 20:48:24.298833 2208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:24.316256 2208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:24.318706 2208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:24.318868 2208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:24.323047 2208 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46867
I20260619 20:48:24.323133 2276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46867 every 8 connection(s)
I20260619 20:48:24.323464 2208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:24.324357 2277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:24.326782 2277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb: Bootstrap starting.
I20260619 20:48:24.327385 2277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:24.327679 2277 log.cc:824] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:24.328400 2277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb: No bootstrap required, opened a new log
I20260619 20:48:24.330062 2277 raft_consensus.cc:348] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46867 } }
I20260619 20:48:24.330201 2277 raft_consensus.cc:374] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:24.330224 2277 raft_consensus.cc:729] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db70280a31a248f5a9b6ee5adcc65eeb, State: Initialized, Role: FOLLOWER
I20260619 20:48:24.330370 2277 consensus_queue.cc:260] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46867 } }
I20260619 20:48:24.330497 2277 raft_consensus.cc:388] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:24.330524 2277 raft_consensus.cc:482] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:24.330590 2277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:24.331140 2277 raft_consensus.cc:504] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46867 } }
I20260619 20:48:24.331249 2277 leader_election.cc:302] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db70280a31a248f5a9b6ee5adcc65eeb; no voters:
I20260619 20:48:24.331440 2277 leader_election.cc:288] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:24.331501 2280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:24.331702 2277 sys_catalog.cc:564] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:24.331981 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2208
I20260619 20:48:24.332145 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:24.332341 14093 external_mini_cluster.cc:1428] Setting key 82c3b8bfb7c26f772a1aa65562f71d4c
I20260619 20:48:24.332333 2280 raft_consensus.cc:686] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [term 1 LEADER]: Becoming Leader. State: Replica: db70280a31a248f5a9b6ee5adcc65eeb, State: Running, Role: LEADER
I20260619 20:48:24.332429 2280 consensus_queue.cc:237] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46867 } }
I20260619 20:48:24.332978 2282 sys_catalog.cc:455] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader db70280a31a248f5a9b6ee5adcc65eeb. Latest consensus state: current_term: 1 leader_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46867 } } }
I20260619 20:48:24.333091 2282 sys_catalog.cc:458] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:24.333348 2281 sys_catalog.cc:455] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db70280a31a248f5a9b6ee5adcc65eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46867 } } }
I20260619 20:48:24.333428 2281 sys_catalog.cc:458] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb [sys.catalog]: This master's current role is: LEADER
W20260619 20:48:24.333750 2296 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:24.333848 2296 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:24.333949 2297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:24.334125 2297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:24.338899 2297 catalog_manager.cc:1347] Generated new cluster ID: 900f5a7d36cc4f98b61b621efe569800
I20260619 20:48:24.338963 2297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:24 dist-test-slave-n2td krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:48:24.344534 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:24.332959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47407 (local address 127.0.0.1:46867)
0619 20:48:24.333132 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:24.333142 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:24.333412 (+ 270us) server_negotiation.cc:407] Connection header received
0619 20:48:24.333555 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:24.333563 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:24.333639 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:24.333785 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:24.334625 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:24.335725 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:24.336823 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:24.337242 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:24.337323 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:24.340157 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:24.340161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:24.340161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:24.340172 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:24.342848 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:24.343259 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:24.343263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:24.343263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:24.343390 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:24.343616 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:24.343617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:24.343617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:24.343913 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:24.344049 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:24.344182 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 20:48:24.344314 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260619 20:48:24.345606 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:24.347003 2297 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:24.347733 2297 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:24.353854 2297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db70280a31a248f5a9b6ee5adcc65eeb: Generated new TSK 0
I20260619 20:48:24.354085 2297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:24.354740 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:24.347113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41993 (local address 127.0.0.1:46867)
0619 20:48:24.347146 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:24.347148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:24.347296 (+ 148us) server_negotiation.cc:407] Connection header received
0619 20:48:24.347340 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:24.347341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:24.347350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:24.347417 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:24.348034 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:24.348827 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:24.349865 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:24.350172 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:24.350230 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:24.351743 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:24.351745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:24.351745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:24.351752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:24.353392 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:24.353781 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:24.353782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:24.353783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:24.353859 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:24.354115 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:24.354116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:24.354117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:24.354242 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:24.354357 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:24.354531 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 20:48:24.354633 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:48:24.356650 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2208
2026-06-19T20:48:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:24 dist-test-slave-n2td krb5kdc[2308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:24 dist-test-slave-n2td krb5kdc[2308](info): set up 2 sockets
Jun 19 20:48:24 dist-test-slave-n2td krb5kdc[2308](info): commencing operation
krb5kdc: starting...
W20260619 20:48:25.405366 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:25 dist-test-slave-n2td krb5kdc[2308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:25.527559 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:25.611243 2322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:25.611480 2322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:25.611521 2322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:25.612725 2322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:25.612797 2322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:25.612823 2322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:25.612852 2322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:25.612900 2322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:25.614419 2322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:25.614872 2322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:25.615170 2322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:25.620129 2330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:25.620244 2322 server_base.cc:1034] running on GCE node
W20260619 20:48:25.620131 2328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:25.620147 2327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:25.620626 2322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:25.620961 2322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:25.622110 2322 hybrid_clock.cc:648] HybridClock initialized: now 1781902105622090 us; error 35 us; skew 500 ppm
Jun 19 20:48:25 dist-test-slave-n2td krb5kdc[2308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902105, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:25.624640 2322 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:25.625128 2322 webserver.cc:460] Webserver started at http://127.0.0.1:41517/ using document root <none> and password file <none>
I20260619 20:48:25.625355 2322 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:25.625423 2322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:25.625579 2322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:25.626727 2322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "d37288b3f4474fef9f72ae3cb77729cc"
format_stamp: "Formatted at 2026-06-19 20:48:25 on dist-test-slave-n2td"
server_key: "3d546dcc8b08f76871801b0b6911e1ea"
server_key_iv: "099da8d3e9a84da9c8a74ffb18de4354"
server_key_version: "encryptionkey@0"
I20260619 20:48:25.627100 2322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "d37288b3f4474fef9f72ae3cb77729cc"
format_stamp: "Formatted at 2026-06-19 20:48:25 on dist-test-slave-n2td"
server_key: "3d546dcc8b08f76871801b0b6911e1ea"
server_key_iv: "099da8d3e9a84da9c8a74ffb18de4354"
server_key_version: "encryptionkey@0"
I20260619 20:48:25.628437 2322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:25.629209 2337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:25.629392 2322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:25.629492 2322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "d37288b3f4474fef9f72ae3cb77729cc"
format_stamp: "Formatted at 2026-06-19 20:48:25 on dist-test-slave-n2td"
server_key: "3d546dcc8b08f76871801b0b6911e1ea"
server_key_iv: "099da8d3e9a84da9c8a74ffb18de4354"
server_key_version: "encryptionkey@0"
I20260619 20:48:25.629591 2322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:25.645475 2322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:25.647415 2322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:25.647588 2322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:25.651798 2322 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42745
I20260619 20:48:25.651839 2389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42745 every 8 connection(s)
I20260619 20:48:25.652202 2322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:25.653048 2390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:25.655388 2390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc: Bootstrap starting.
I20260619 20:48:25.655907 2390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:25.656147 2390 log.cc:824] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:25.656790 2390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc: No bootstrap required, opened a new log
I20260619 20:48:25.658341 2390 raft_consensus.cc:348] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37288b3f4474fef9f72ae3cb77729cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42745 } }
I20260619 20:48:25.658555 2390 raft_consensus.cc:374] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:25.658584 2390 raft_consensus.cc:729] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37288b3f4474fef9f72ae3cb77729cc, State: Initialized, Role: FOLLOWER
I20260619 20:48:25.658684 2390 consensus_queue.cc:260] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37288b3f4474fef9f72ae3cb77729cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42745 } }
I20260619 20:48:25.658737 2390 raft_consensus.cc:388] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:25.658757 2390 raft_consensus.cc:482] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:25.658824 2390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:25.659330 2390 raft_consensus.cc:504] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37288b3f4474fef9f72ae3cb77729cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42745 } }
I20260619 20:48:25.659449 2390 leader_election.cc:302] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d37288b3f4474fef9f72ae3cb77729cc; no voters:
I20260619 20:48:25.659626 2390 leader_election.cc:288] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:25.659659 2393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:25.659874 2390 sys_catalog.cc:564] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:25.659911 2393 raft_consensus.cc:686] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [term 1 LEADER]: Becoming Leader. State: Replica: d37288b3f4474fef9f72ae3cb77729cc, State: Running, Role: LEADER
I20260619 20:48:25.659993 2393 consensus_queue.cc:237] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37288b3f4474fef9f72ae3cb77729cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42745 } }
I20260619 20:48:25.660401 2395 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d37288b3f4474fef9f72ae3cb77729cc. Latest consensus state: current_term: 1 leader_uuid: "d37288b3f4474fef9f72ae3cb77729cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37288b3f4474fef9f72ae3cb77729cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42745 } } }
I20260619 20:48:25.660526 2395 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:25.660542 2394 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d37288b3f4474fef9f72ae3cb77729cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37288b3f4474fef9f72ae3cb77729cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42745 } } }
I20260619 20:48:25.660629 2394 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:25.660967 2402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:25.661520 2402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:25.662305 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2322
I20260619 20:48:25.662504 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:25.662719 14093 external_mini_cluster.cc:1428] Setting key 177e47e6a122dd425baa3121433bcbc0
I20260619 20:48:25.663221 2402 catalog_manager.cc:1347] Generated new cluster ID: 90760f0d3a7a4f4eaffe1916af687524
I20260619 20:48:25.663265 2402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:25.671715 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:25.663168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59805 (local address 127.0.0.1:42745)
0619 20:48:25.663310 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:25.663318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:25.666829 (+ 3511us) server_negotiation.cc:407] Connection header received
0619 20:48:25.667001 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:25.667007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:25.667049 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:25.667189 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:25.667428 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:25.668731 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:25.669570 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:25.669989 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:25.670109 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:25.670913 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:25.670917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:25.670917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:25.671095 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:25.671126 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:25.671136 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:25.671215 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:25.671336 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 20:48:25.671436 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
I20260619 20:48:25.672797 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:25.677062 2402 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:25.677556 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:25.674154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57833 (local address 127.0.0.1:42745)
0619 20:48:25.674210 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:25.674211 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:25.674695 (+ 484us) server_negotiation.cc:407] Connection header received
0619 20:48:25.674772 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:25.674773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:25.674784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:25.674840 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:25.675013 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:25.675518 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:25.676323 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:25.676522 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:25.676596 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:25.677291 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:25.677292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:25.677292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:25.677297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:25.677310 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:25.677314 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:25.677367 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:25.677436 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 20:48:25.677458 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:48:25.677809 2402 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:25.682945 2402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d37288b3f4474fef9f72ae3cb77729cc: Generated new TSK 0
I20260619 20:48:25.683142 2402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:25.685549 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2322
2026-06-19T20:48:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:25 dist-test-slave-n2td krb5kdc[2421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:25 dist-test-slave-n2td krb5kdc[2421](info): set up 2 sockets
Jun 19 20:48:25 dist-test-slave-n2td krb5kdc[2421](info): commencing operation
krb5kdc: starting...
W20260619 20:48:26.721397 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:26 dist-test-slave-n2td krb5kdc[2421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:26.846263 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:26.928844 2434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:26.929064 2434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:26.929087 2434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:26.930305 2434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:26.930351 2434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:26.930366 2434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:26.930378 2434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:26.930392 2434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:26.931931 2434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:26.932190 2434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:26.932418 2434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:26.937500 2440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:26.937492 2442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:26.937497 2439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:26.937852 2434 server_base.cc:1034] running on GCE node
I20260619 20:48:26.938102 2434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:26.938403 2434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:26.939596 2434 hybrid_clock.cc:648] HybridClock initialized: now 1781902106939583 us; error 30 us; skew 500 ppm
Jun 19 20:48:26 dist-test-slave-n2td krb5kdc[2421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:26.944805 2434 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:26.945286 2434 webserver.cc:460] Webserver started at http://127.0.0.1:45539/ using document root <none> and password file <none>
I20260619 20:48:26.945541 2434 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:26.945612 2434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:26.945772 2434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:26.946916 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d"
format_stamp: "Formatted at 2026-06-19 20:48:26 on dist-test-slave-n2td"
server_key: "cecd3b4dc74c90a936553358cc452ac5"
server_key_iv: "59f9203cb78d0b7c4bd2ce621f6ce321"
server_key_version: "encryptionkey@0"
I20260619 20:48:26.947288 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d"
format_stamp: "Formatted at 2026-06-19 20:48:26 on dist-test-slave-n2td"
server_key: "cecd3b4dc74c90a936553358cc452ac5"
server_key_iv: "59f9203cb78d0b7c4bd2ce621f6ce321"
server_key_version: "encryptionkey@0"
I20260619 20:48:26.948653 2434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:26.949438 2450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:26.949617 2434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:26.949738 2434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d"
format_stamp: "Formatted at 2026-06-19 20:48:26 on dist-test-slave-n2td"
server_key: "cecd3b4dc74c90a936553358cc452ac5"
server_key_iv: "59f9203cb78d0b7c4bd2ce621f6ce321"
server_key_version: "encryptionkey@0"
I20260619 20:48:26.949813 2434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:26.953163 2434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:26.955205 2434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:26.955359 2434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:26.959277 2434 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46757
I20260619 20:48:26.959306 2503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46757 every 8 connection(s)
I20260619 20:48:26.959632 2434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:26.960459 2504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:26.960836 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2434
I20260619 20:48:26.960958 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:26.961158 14093 external_mini_cluster.cc:1428] Setting key e4e71167ed66ba831c7f1972e66f00ef
I20260619 20:48:26.963594 2504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d: Bootstrap starting.
I20260619 20:48:26.963666 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:26.961688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50751 (local address 127.0.0.1:46757)
0619 20:48:26.961876 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:26.961885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:26.961921 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:48:26.962332 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:26.962338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:26.962361 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:26.962553 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:26.963187 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:26.963191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:26.963191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:26.963194 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:26.963229 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:26.963238 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:26.963294 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:26.963388 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:48:26.963440 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260619 20:48:26.964318 2504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:26.964679 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:26.964666 2504 log.cc:824] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:26.965410 2504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d: No bootstrap required, opened a new log
I20260619 20:48:26.967394 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:26.966281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53077 (local address 127.0.0.1:46757)
0619 20:48:26.966321 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:26.966323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:26.966467 (+ 144us) server_negotiation.cc:407] Connection header received
0619 20:48:26.966520 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:26.966521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:26.966525 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:26.966566 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:26.967112 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:26.967113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:26.967113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:26.967118 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:26.967133 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:26.967138 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:26.967216 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:26.967293 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:48:26.967304 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:48:26.968196 2504 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46757 } }
I20260619 20:48:26.968331 2504 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:26.968353 2504 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6b1c555d2b4adfbb8c502bbd1e207d, State: Initialized, Role: FOLLOWER
I20260619 20:48:26.968498 2504 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46757 } }
I20260619 20:48:26.968587 2504 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:26.968640 2504 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:26.968693 2504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:26.969233 2504 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46757 } }
I20260619 20:48:26.969344 2504 leader_election.cc:302] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce6b1c555d2b4adfbb8c502bbd1e207d; no voters:
I20260619 20:48:26.969498 2504 leader_election.cc:288] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:26.969544 2515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:26.969738 2515 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [term 1 LEADER]: Becoming Leader. State: Replica: ce6b1c555d2b4adfbb8c502bbd1e207d, State: Running, Role: LEADER
I20260619 20:48:26.969762 2504 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:26.969830 2515 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46757 } }
I20260619 20:48:26.970494 2517 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce6b1c555d2b4adfbb8c502bbd1e207d. Latest consensus state: current_term: 1 leader_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46757 } } }
I20260619 20:48:26.970582 2517 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:26.970798 2516 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6b1c555d2b4adfbb8c502bbd1e207d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46757 } } }
I20260619 20:48:26.970872 2525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:26.970934 2516 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:26.971382 2525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:26.972896 2525 catalog_manager.cc:1347] Generated new cluster ID: a612010db47c4cac92bec4933b4a6f42
I20260619 20:48:26.972945 2525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:26.983507 2525 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:26.983723 2525 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:26.999528 2525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce6b1c555d2b4adfbb8c502bbd1e207d: Generated new TSK 0
I20260619 20:48:26.999755 2525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:27.009050 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2434
2026-06-19T20:48:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:27 dist-test-slave-n2td krb5kdc[2537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:27 dist-test-slave-n2td krb5kdc[2537](info): set up 2 sockets
Jun 19 20:48:27 dist-test-slave-n2td krb5kdc[2537](info): commencing operation
krb5kdc: starting...
W20260619 20:48:28.050884 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:28 dist-test-slave-n2td krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:28.168951 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:28.250795 2550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:28.251037 2550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:28.251087 2550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:28.252272 2550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:28.252347 2550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:28.252374 2550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:28.252406 2550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:28.252432 2550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:28.254016 2550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:28.254308 2550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:28.254573 2550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:28.259323 2558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:28.259289 2555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:28.259286 2556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:28.260047 2550 server_base.cc:1034] running on GCE node
I20260619 20:48:28.260303 2550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:28.260645 2550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:28.261853 2550 hybrid_clock.cc:648] HybridClock initialized: now 1781902108261828 us; error 34 us; skew 500 ppm
Jun 19 20:48:28 dist-test-slave-n2td krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:28.264549 2550 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:28.265043 2550 webserver.cc:460] Webserver started at http://127.0.0.1:40345/ using document root <none> and password file <none>
I20260619 20:48:28.265318 2550 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:28.265398 2550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:28.265619 2550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:28.266865 2550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "d9131c305dfa4ff987f4098bfe4f894f"
format_stamp: "Formatted at 2026-06-19 20:48:28 on dist-test-slave-n2td"
server_key: "9dd2893fe333210fb7eac8bb76875643"
server_key_iv: "bef86f92d08fa831cd24f40f67f55595"
server_key_version: "encryptionkey@0"
I20260619 20:48:28.267254 2550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "d9131c305dfa4ff987f4098bfe4f894f"
format_stamp: "Formatted at 2026-06-19 20:48:28 on dist-test-slave-n2td"
server_key: "9dd2893fe333210fb7eac8bb76875643"
server_key_iv: "bef86f92d08fa831cd24f40f67f55595"
server_key_version: "encryptionkey@0"
I20260619 20:48:28.268711 2550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:28.269526 2566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:28.269712 2550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:28.269824 2550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "d9131c305dfa4ff987f4098bfe4f894f"
format_stamp: "Formatted at 2026-06-19 20:48:28 on dist-test-slave-n2td"
server_key: "9dd2893fe333210fb7eac8bb76875643"
server_key_iv: "bef86f92d08fa831cd24f40f67f55595"
server_key_version: "encryptionkey@0"
I20260619 20:48:28.269912 2550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:28.289585 2550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:28.291664 2550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:28.291847 2550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:28.296103 2550 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45203
I20260619 20:48:28.296168 2620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45203 every 8 connection(s)
I20260619 20:48:28.296583 2550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:28.297487 2621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:28.299785 2621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f: Bootstrap starting.
I20260619 20:48:28.300386 2621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:28.300679 2621 log.cc:824] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:28.301370 2621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f: No bootstrap required, opened a new log
I20260619 20:48:28.303153 2621 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9131c305dfa4ff987f4098bfe4f894f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45203 } }
I20260619 20:48:28.303365 2621 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:28.303395 2621 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9131c305dfa4ff987f4098bfe4f894f, State: Initialized, Role: FOLLOWER
I20260619 20:48:28.303551 2621 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9131c305dfa4ff987f4098bfe4f894f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45203 } }
I20260619 20:48:28.303658 2621 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:28.303704 2621 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:28.303755 2621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:28.303843 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2550
I20260619 20:48:28.304003 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:28.304219 14093 external_mini_cluster.cc:1428] Setting key b7f8a315c9190b259dc0e2915cad7c69
I20260619 20:48:28.304430 2621 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9131c305dfa4ff987f4098bfe4f894f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45203 } }
I20260619 20:48:28.304543 2621 leader_election.cc:302] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9131c305dfa4ff987f4098bfe4f894f; no voters:
I20260619 20:48:28.304747 2621 leader_election.cc:288] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:28.304768 2624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:28.304875 2624 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [term 1 LEADER]: Becoming Leader. State: Replica: d9131c305dfa4ff987f4098bfe4f894f, State: Running, Role: LEADER
I20260619 20:48:28.305006 2624 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9131c305dfa4ff987f4098bfe4f894f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45203 } }
I20260619 20:48:28.305193 2621 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:28.305327 2624 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9131c305dfa4ff987f4098bfe4f894f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9131c305dfa4ff987f4098bfe4f894f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45203 } } }
I20260619 20:48:28.305500 2624 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:28.305672 2624 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9131c305dfa4ff987f4098bfe4f894f. Latest consensus state: current_term: 1 leader_uuid: "d9131c305dfa4ff987f4098bfe4f894f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9131c305dfa4ff987f4098bfe4f894f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45203 } } }
I20260619 20:48:28.305795 2624 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:28.306190 2630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:28.306763 2630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:28.308348 2630 catalog_manager.cc:1347] Generated new cluster ID: 43b558abf9ec47f3a31d218f03fa05f6
I20260619 20:48:28.308403 2630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:28 dist-test-slave-n2td krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:48:28.313378 2630 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:28.314193 2630 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:28.316408 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:28.305033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:37703 (local address 127.0.0.1:45203)
0619 20:48:28.305185 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:28.305195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:28.305228 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:48:28.305357 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:28.305366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:28.305426 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:28.306401 (+ 975us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:28.306529 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:28.308059 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:28.308943 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:28.309361 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:28.309501 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:28.311924 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:28.311928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:28.311928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:28.311948 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:28.314710 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:28.315188 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:28.315191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:28.315192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:28.315296 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:28.315539 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:28.315539 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:28.315540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:28.315813 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:28.315951 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:28.316106 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 20:48:28.316199 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260619 20:48:28.317286 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:28.320003 2630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9131c305dfa4ff987f4098bfe4f894f: Generated new TSK 0
I20260619 20:48:28.320159 2630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:28.329166 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:28.318745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:50553 (local address 127.0.0.1:45203)
0619 20:48:28.318779 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:28.318781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:28.322656 (+ 3875us) server_negotiation.cc:407] Connection header received
0619 20:48:28.322687 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:28.322688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:28.322701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:28.322768 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:28.323286 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:28.324095 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:28.324871 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:28.325203 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:28.325276 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:28.326557 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:28.326558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:28.326558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:28.326565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:28.328058 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:28.328410 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:28.328411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:28.328411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:28.328471 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:28.328678 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:28.328678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:28.328679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:28.328805 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:28.328899 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:28.329008 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 20:48:28.329071 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:48:28.331146 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2550
W20260619 20:48:28.335465 2535 connection.cc:535] client connection to 127.0.0.1:45203 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45203): Connection reset by peer (error 104)
2026-06-19T20:48:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:28 dist-test-slave-n2td krb5kdc[2653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:28 dist-test-slave-n2td krb5kdc[2653](info): set up 2 sockets
Jun 19 20:48:28 dist-test-slave-n2td krb5kdc[2653](info): commencing operation
krb5kdc: starting...
W20260619 20:48:29.369376 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:29 dist-test-slave-n2td krb5kdc[2653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:29.487527 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:29.568128 2666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:29.568348 2666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:29.568372 2666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:29.569522 2666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:29.569576 2666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:29.569589 2666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:29.569602 2666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:29.569656 2666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:29.571105 2666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:29.571355 2666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:29.571576 2666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:29.576382 2674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:29.576416 2671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:29.576382 2672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:29.576578 2666 server_base.cc:1034] running on GCE node
I20260619 20:48:29.576893 2666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:29.577312 2666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:29.578498 2666 hybrid_clock.cc:648] HybridClock initialized: now 1781902109578473 us; error 37 us; skew 500 ppm
Jun 19 20:48:29 dist-test-slave-n2td krb5kdc[2653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902109, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:29.580910 2666 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:29.581341 2666 webserver.cc:460] Webserver started at http://127.0.0.1:36677/ using document root <none> and password file <none>
I20260619 20:48:29.581558 2666 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:29.581627 2666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:29.581771 2666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:29.582904 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "11cb0636b1dc4be980832e75587bb3dd"
format_stamp: "Formatted at 2026-06-19 20:48:29 on dist-test-slave-n2td"
server_key: "1a6b6dd3c54caec4824edca1358ee35b"
server_key_iv: "84c35a9199173b33fe55edb40aafbf2b"
server_key_version: "encryptionkey@0"
I20260619 20:48:29.583262 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "11cb0636b1dc4be980832e75587bb3dd"
format_stamp: "Formatted at 2026-06-19 20:48:29 on dist-test-slave-n2td"
server_key: "1a6b6dd3c54caec4824edca1358ee35b"
server_key_iv: "84c35a9199173b33fe55edb40aafbf2b"
server_key_version: "encryptionkey@0"
I20260619 20:48:29.584571 2666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:29.585281 2681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:29.585450 2666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:29.585551 2666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "11cb0636b1dc4be980832e75587bb3dd"
format_stamp: "Formatted at 2026-06-19 20:48:29 on dist-test-slave-n2td"
server_key: "1a6b6dd3c54caec4824edca1358ee35b"
server_key_iv: "84c35a9199173b33fe55edb40aafbf2b"
server_key_version: "encryptionkey@0"
I20260619 20:48:29.585624 2666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:29.596571 2666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:29.598404 2666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:29.598603 2666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:29.604087 2666 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34861
I20260619 20:48:29.604157 2734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34861 every 8 connection(s)
I20260619 20:48:29.604569 2666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:29.605670 2735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:29.608376 2735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd: Bootstrap starting.
I20260619 20:48:29.608948 2735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:29.609248 2735 log.cc:824] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:29.609963 2735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd: No bootstrap required, opened a new log
I20260619 20:48:29.611609 2735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cb0636b1dc4be980832e75587bb3dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34861 } }
I20260619 20:48:29.611768 2735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:29.611819 2735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11cb0636b1dc4be980832e75587bb3dd, State: Initialized, Role: FOLLOWER
I20260619 20:48:29.611961 2735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cb0636b1dc4be980832e75587bb3dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34861 } }
I20260619 20:48:29.612051 2735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:29.612092 2735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:29.612143 2735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:29.612336 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2666
I20260619 20:48:29.612453 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:29.612651 14093 external_mini_cluster.cc:1428] Setting key 304147f9ef6684eea864f68b1fa4c971
I20260619 20:48:29.612776 2735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cb0636b1dc4be980832e75587bb3dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34861 } }
I20260619 20:48:29.612897 2735 leader_election.cc:302] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11cb0636b1dc4be980832e75587bb3dd; no voters:
I20260619 20:48:29.613085 2735 leader_election.cc:288] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:29.613106 2738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:29.613318 2738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [term 1 LEADER]: Becoming Leader. State: Replica: 11cb0636b1dc4be980832e75587bb3dd, State: Running, Role: LEADER
I20260619 20:48:29.613360 2735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:29.614539 2738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cb0636b1dc4be980832e75587bb3dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34861 } }
I20260619 20:48:29.615121 2742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11cb0636b1dc4be980832e75587bb3dd. Latest consensus state: current_term: 1 leader_uuid: "11cb0636b1dc4be980832e75587bb3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cb0636b1dc4be980832e75587bb3dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34861 } } }
I20260619 20:48:29.615211 2742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:29.615401 2755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:29.615398 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11cb0636b1dc4be980832e75587bb3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11cb0636b1dc4be980832e75587bb3dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34861 } } }
I20260619 20:48:29.615492 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:29.615899 2755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:29.617484 2755 catalog_manager.cc:1347] Generated new cluster ID: df96197539dc4f52ace3b912e3d0815c
I20260619 20:48:29.617534 2755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:29.619256 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:29.613139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:52475 (local address 127.0.0.1:34861)
0619 20:48:29.613695 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:29.613704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:29.613734 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:48:29.613865 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:29.613871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:29.613944 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:29.614108 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:29.614508 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:29.616018 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:29.617001 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:29.617593 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:29.617702 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:29.618448 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:29.618453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:29.618453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:29.618671 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:29.618709 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:29.618723 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:29.618813 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:29.619003 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 20:48:29.619034 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":76,"threads_started":1}
I20260619 20:48:29.620100 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:29.625077 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:29.621482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:47261 (local address 127.0.0.1:34861)
0619 20:48:29.621561 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:29.621562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:29.621690 (+ 128us) server_negotiation.cc:407] Connection header received
0619 20:48:29.621752 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:29.621753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:29.621763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:29.621821 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:29.621939 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:29.622723 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:29.623736 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:29.624035 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:29.624093 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:29.624828 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:29.624829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:29.624829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:29.624833 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:29.624847 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:29.624851 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:29.624910 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:29.624990 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:48:29.625002 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:48:29.626807 2755 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:29.627532 2755 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:29.639325 2755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11cb0636b1dc4be980832e75587bb3dd: Generated new TSK 0
I20260619 20:48:29.639540 2755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:29.645072 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2666
W20260619 20:48:29.649340 2651 connection.cc:535] client connection to 127.0.0.1:34861 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34861): Connection reset by peer (error 104)
2026-06-19T20:48:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:29 dist-test-slave-n2td krb5kdc[2766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:29 dist-test-slave-n2td krb5kdc[2766](info): set up 2 sockets
Jun 19 20:48:29 dist-test-slave-n2td krb5kdc[2766](info): commencing operation
krb5kdc: starting...
W20260619 20:48:30.693382 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:30 dist-test-slave-n2td krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:30.813719 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:30.893513 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:30.893710 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:30.893733 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:30.894830 2779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:30.894866 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:30.894881 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:30.894893 2779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:30.894905 2779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:30.896286 2779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:30.896481 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:30.896651 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:30.901404 2784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:30.901425 2787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:30.901530 2779 server_base.cc:1034] running on GCE node
W20260619 20:48:30.901402 2785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:30.901962 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:30.902261 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:30.903451 2779 hybrid_clock.cc:648] HybridClock initialized: now 1781902110903417 us; error 58 us; skew 500 ppm
Jun 19 20:48:30 dist-test-slave-n2td krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:30.905905 2779 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:30.906335 2779 webserver.cc:460] Webserver started at http://127.0.0.1:40975/ using document root <none> and password file <none>
I20260619 20:48:30.906620 2779 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:30.906692 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:30.906852 2779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:30.907979 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "7ba6453d46f644c59156a24eadca4c24"
format_stamp: "Formatted at 2026-06-19 20:48:30 on dist-test-slave-n2td"
server_key: "4a2e137d84044039b955013a951d9eda"
server_key_iv: "76328dd003b84d9c499f09aa2a133860"
server_key_version: "encryptionkey@0"
I20260619 20:48:30.908357 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "7ba6453d46f644c59156a24eadca4c24"
format_stamp: "Formatted at 2026-06-19 20:48:30 on dist-test-slave-n2td"
server_key: "4a2e137d84044039b955013a951d9eda"
server_key_iv: "76328dd003b84d9c499f09aa2a133860"
server_key_version: "encryptionkey@0"
I20260619 20:48:30.909785 2779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:30.910528 2794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:30.910735 2779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:30.910843 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "7ba6453d46f644c59156a24eadca4c24"
format_stamp: "Formatted at 2026-06-19 20:48:30 on dist-test-slave-n2td"
server_key: "4a2e137d84044039b955013a951d9eda"
server_key_iv: "76328dd003b84d9c499f09aa2a133860"
server_key_version: "encryptionkey@0"
I20260619 20:48:30.910938 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:30.914238 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:30.916604 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:30.916759 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:30.920708 2779 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33637
I20260619 20:48:30.920765 2846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33637 every 8 connection(s)
I20260619 20:48:30.921106 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:30.921968 2847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:30.924203 2847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24: Bootstrap starting.
I20260619 20:48:30.924713 2847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:30.924948 2847 log.cc:824] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:30.925588 2847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24: No bootstrap required, opened a new log
I20260619 20:48:30.927201 2847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6453d46f644c59156a24eadca4c24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260619 20:48:30.927335 2847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:30.927357 2847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba6453d46f644c59156a24eadca4c24, State: Initialized, Role: FOLLOWER
I20260619 20:48:30.927490 2847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6453d46f644c59156a24eadca4c24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260619 20:48:30.927569 2847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:30.927595 2847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:30.927644 2847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:30.928118 2847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6453d46f644c59156a24eadca4c24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260619 20:48:30.928215 2847 leader_election.cc:302] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ba6453d46f644c59156a24eadca4c24; no voters:
I20260619 20:48:30.928369 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2779
I20260619 20:48:30.928406 2847 leader_election.cc:288] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:30.928432 2850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:30.928578 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:30.928682 2850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba6453d46f644c59156a24eadca4c24, State: Running, Role: LEADER
I20260619 20:48:30.928746 2847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:30.928761 2850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6453d46f644c59156a24eadca4c24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260619 20:48:30.928797 14093 external_mini_cluster.cc:1428] Setting key 60043957ae2e6a13937f2b10bf37b4f0
I20260619 20:48:30.929055 2850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba6453d46f644c59156a24eadca4c24. Latest consensus state: current_term: 1 leader_uuid: "7ba6453d46f644c59156a24eadca4c24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6453d46f644c59156a24eadca4c24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } } }
I20260619 20:48:30.929128 2850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:30.929524 2851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ba6453d46f644c59156a24eadca4c24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6453d46f644c59156a24eadca4c24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } } }
I20260619 20:48:30.929589 2860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:30.929665 2851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:30.930229 2860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:30.932005 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:30.929501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:56295 (local address 127.0.0.1:33637)
0619 20:48:30.929741 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:30.929751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:30.930043 (+ 292us) server_negotiation.cc:407] Connection header received
0619 20:48:30.930172 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:30.930180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:30.930213 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:30.930363 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:30.931539 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:30.931542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:30.931542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:30.931547 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:30.931617 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:30.931632 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:30.931719 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:30.931792 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 20:48:30.931835 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
I20260619 20:48:30.932060 2860 catalog_manager.cc:1347] Generated new cluster ID: 7c328f65d85b4f7fbf552dde77e8fd0b
I20260619 20:48:30.932394 2860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:30.933202 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:30.936007 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:30.934801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:53777 (local address 127.0.0.1:33637)
0619 20:48:30.934844 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:30.934846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:30.935057 (+ 211us) server_negotiation.cc:407] Connection header received
0619 20:48:30.935113 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:30.935114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:30.935119 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:30.935178 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:30.935783 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:30.935784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:30.935785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:30.935789 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:30.935803 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:30.935807 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:30.935868 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:30.935929 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 20:48:30.935943 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:48:30.938738 2860 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:30.938884 2860 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:30.941848 2860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ba6453d46f644c59156a24eadca4c24: Generated new TSK 0
I20260619 20:48:30.942009 2860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:30.948235 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2779
2026-06-19T20:48:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:31 dist-test-slave-n2td krb5kdc[2878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:31 dist-test-slave-n2td krb5kdc[2878](info): set up 2 sockets
Jun 19 20:48:31 dist-test-slave-n2td krb5kdc[2878](info): commencing operation
krb5kdc: starting...
W20260619 20:48:32.001499 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:32 dist-test-slave-n2td krb5kdc[2878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:32.117461 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:32.200359 2892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:32.200551 2892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:32.200572 2892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:32.201722 2892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:32.201761 2892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:32.201776 2892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:32.201789 2892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:32.201802 2892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:32.203248 2892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43049
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:32.203426 2892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:32.203595 2892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:32.208242 2898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:32.208411 2892 server_base.cc:1034] running on GCE node
W20260619 20:48:32.208243 2900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:32.208295 2897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:32.208704 2892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:32.209014 2892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:32.210204 2892 hybrid_clock.cc:648] HybridClock initialized: now 1781902112210201 us; error 39 us; skew 500 ppm
Jun 19 20:48:32 dist-test-slave-n2td krb5kdc[2878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:32.215737 2892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:32.216233 2892 webserver.cc:460] Webserver started at http://127.0.0.1:36067/ using document root <none> and password file <none>
I20260619 20:48:32.216482 2892 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:32.216552 2892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:32.216704 2892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:32.217767 2892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "990fc773e1974149b3b0ca0d47327473"
format_stamp: "Formatted at 2026-06-19 20:48:32 on dist-test-slave-n2td"
server_key: "aa4ef70943acc6757c9fa5646d2e9102"
server_key_iv: "e03e971edf6c002a0d744b411279f591"
server_key_version: "encryptionkey@0"
I20260619 20:48:32.218133 2892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "990fc773e1974149b3b0ca0d47327473"
format_stamp: "Formatted at 2026-06-19 20:48:32 on dist-test-slave-n2td"
server_key: "aa4ef70943acc6757c9fa5646d2e9102"
server_key_iv: "e03e971edf6c002a0d744b411279f591"
server_key_version: "encryptionkey@0"
I20260619 20:48:32.219509 2892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:48:32.220217 2907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:32.220404 2892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:48:32.220507 2892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "990fc773e1974149b3b0ca0d47327473"
format_stamp: "Formatted at 2026-06-19 20:48:32 on dist-test-slave-n2td"
server_key: "aa4ef70943acc6757c9fa5646d2e9102"
server_key_iv: "e03e971edf6c002a0d744b411279f591"
server_key_version: "encryptionkey@0"
I20260619 20:48:32.220587 2892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:32.245875 2892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:32.248332 2892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:32.248493 2892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:32.252642 2892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43049
I20260619 20:48:32.252677 2959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43049 every 8 connection(s)
I20260619 20:48:32.253074 2892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:32.253906 2960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:32.256215 2960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473: Bootstrap starting.
I20260619 20:48:32.256786 2960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:32.257061 2960 log.cc:824] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:32.257731 2960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473: No bootstrap required, opened a new log
I20260619 20:48:32.259377 2960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990fc773e1974149b3b0ca0d47327473" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43049 } }
I20260619 20:48:32.259514 2960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:32.259560 2960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990fc773e1974149b3b0ca0d47327473, State: Initialized, Role: FOLLOWER
I20260619 20:48:32.259713 2960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990fc773e1974149b3b0ca0d47327473" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43049 } }
I20260619 20:48:32.259776 2960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:32.259816 2960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:32.259865 2960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:32.260383 2960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990fc773e1974149b3b0ca0d47327473" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43049 } }
I20260619 20:48:32.260491 2960 leader_election.cc:302] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 990fc773e1974149b3b0ca0d47327473; no voters:
I20260619 20:48:32.260685 2960 leader_election.cc:288] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:32.260713 2964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:32.260882 2964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [term 1 LEADER]: Becoming Leader. State: Replica: 990fc773e1974149b3b0ca0d47327473, State: Running, Role: LEADER
I20260619 20:48:32.260938 2960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:32.261010 2964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990fc773e1974149b3b0ca0d47327473" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43049 } }
I20260619 20:48:32.261402 2966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 990fc773e1974149b3b0ca0d47327473. Latest consensus state: current_term: 1 leader_uuid: "990fc773e1974149b3b0ca0d47327473" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990fc773e1974149b3b0ca0d47327473" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43049 } } }
I20260619 20:48:32.261480 2966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:32.261548 2965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "990fc773e1974149b3b0ca0d47327473" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990fc773e1974149b3b0ca0d47327473" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43049 } } }
I20260619 20:48:32.261634 2965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:32.261935 2972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:32.262336 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 2892
I20260619 20:48:32.262465 2972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:32.262531 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:32.262761 14093 external_mini_cluster.cc:1428] Setting key 8064dd236986ec5f56b58f4e4704bb28
I20260619 20:48:32.264204 2972 catalog_manager.cc:1347] Generated new cluster ID: 93871300666f4dd4b477f91492ec3d91
I20260619 20:48:32.264256 2972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:32 dist-test-slave-n2td krb5kdc[2878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:48:32.272873 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:32.263225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:42211 (local address 127.0.0.1:43049)
0619 20:48:32.263383 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:32.263392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:32.263586 (+ 194us) server_negotiation.cc:407] Connection header received
0619 20:48:32.263704 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:32.263711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:32.263770 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:32.263919 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:32.264650 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:32.265648 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:32.266446 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:32.266923 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:32.267072 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:32.269436 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:32.269440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:32.269440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:32.269452 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:32.271216 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:32.271632 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:32.271635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:32.271635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:32.271733 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:32.271969 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:32.271970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:32.271971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:32.272215 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:32.272402 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:32.272492 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 20:48:32.272703 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260619 20:48:32.273816 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:32.281289 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:32.275218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:38059 (local address 127.0.0.1:43049)
0619 20:48:32.275259 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:32.275260 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:32.275394 (+ 134us) server_negotiation.cc:407] Connection header received
0619 20:48:32.275448 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:32.275449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:32.275460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:32.275514 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:32.276022 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:32.276518 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:32.277247 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:32.277471 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:32.277546 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:32.278790 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:32.278792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:32.278793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:32.278799 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:32.280137 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:32.280499 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:32.280500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:32.280500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:32.280551 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:32.280771 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:32.280772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:32.280773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:32.280899 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:32.281005 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:32.281115 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 20:48:32.281175 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:48:32.306421 2972 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:32.307008 2972 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:32.312376 2972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 990fc773e1974149b3b0ca0d47327473: Generated new TSK 0
I20260619 20:48:32.312587 2972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:32.318269 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 2892
W20260619 20:48:32.322937 2876 connection.cc:535] client connection to 127.0.0.1:43049 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43049): Connection reset by peer (error 104)
2026-06-19T20:48:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1370 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:32 dist-test-slave-n2td krb5kdc[2993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:32 dist-test-slave-n2td krb5kdc[2993](info): set up 2 sockets
Jun 19 20:48:32 dist-test-slave-n2td krb5kdc[2993](info): commencing operation
krb5kdc: starting...
W20260619 20:48:33.381524 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:33 dist-test-slave-n2td krb5kdc[2993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:33.501137 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39991
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:33.581349 3007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:33.581578 3007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:33.581614 3007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:33.582818 3007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:33.582903 3007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:33.582952 3007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:33.582983 3007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:33.583011 3007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:33.584476 3007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39991
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:33.584760 3007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:33.584982 3007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:33.589757 3015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:33.589787 3013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:33.589967 3007 server_base.cc:1034] running on GCE node
W20260619 20:48:33.589764 3012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:33.590282 3007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:33.590615 3007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:33.591773 3007 hybrid_clock.cc:648] HybridClock initialized: now 1781902113591760 us; error 33 us; skew 500 ppm
Jun 19 20:48:33 dist-test-slave-n2td krb5kdc[2993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902113, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:33.594187 3007 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:33.594645 3007 webserver.cc:460] Webserver started at http://127.0.0.1:37779/ using document root <none> and password file <none>
I20260619 20:48:33.594846 3007 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:33.594885 3007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:33.594985 3007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:33.596042 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "c8495f9effe94961a49f1b80249bd462"
format_stamp: "Formatted at 2026-06-19 20:48:33 on dist-test-slave-n2td"
server_key: "c4e280b83970e9fb82c044fb21f9fc5d"
server_key_iv: "951379e7439a3e7b686ec38854d378b6"
server_key_version: "encryptionkey@0"
I20260619 20:48:33.596355 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "c8495f9effe94961a49f1b80249bd462"
format_stamp: "Formatted at 2026-06-19 20:48:33 on dist-test-slave-n2td"
server_key: "c4e280b83970e9fb82c044fb21f9fc5d"
server_key_iv: "951379e7439a3e7b686ec38854d378b6"
server_key_version: "encryptionkey@0"
I20260619 20:48:33.597627 3007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:33.598457 3022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:33.598644 3007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:33.598731 3007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "c8495f9effe94961a49f1b80249bd462"
format_stamp: "Formatted at 2026-06-19 20:48:33 on dist-test-slave-n2td"
server_key: "c4e280b83970e9fb82c044fb21f9fc5d"
server_key_iv: "951379e7439a3e7b686ec38854d378b6"
server_key_version: "encryptionkey@0"
I20260619 20:48:33.598807 3007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:33.621733 3007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:33.624109 3007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:33.624274 3007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:33.628393 3007 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39991
I20260619 20:48:33.628463 3074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39991 every 8 connection(s)
I20260619 20:48:33.628794 3007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:33.629750 3075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:33.632266 3075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462: Bootstrap starting.
I20260619 20:48:33.632916 3075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:33.633252 3075 log.cc:824] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:33.633991 3075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462: No bootstrap required, opened a new log
I20260619 20:48:33.635959 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3007
I20260619 20:48:33.635890 3075 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8495f9effe94961a49f1b80249bd462" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39991 } }
I20260619 20:48:33.636027 3075 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:33.636096 3075 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8495f9effe94961a49f1b80249bd462, State: Initialized, Role: FOLLOWER
I20260619 20:48:33.636123 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:33.636310 14093 external_mini_cluster.cc:1428] Setting key eec8aa92135ac3d1a8ea6ed10bd3d677
I20260619 20:48:33.636278 3075 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8495f9effe94961a49f1b80249bd462" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39991 } }
I20260619 20:48:33.636360 3075 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:33.636420 3075 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:33.636474 3075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:33.637697 3075 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8495f9effe94961a49f1b80249bd462" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39991 } }
I20260619 20:48:33.637820 3075 leader_election.cc:302] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8495f9effe94961a49f1b80249bd462; no voters:
I20260619 20:48:33.637981 3075 leader_election.cc:288] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:33.638038 3080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:33.638186 3080 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [term 1 LEADER]: Becoming Leader. State: Replica: c8495f9effe94961a49f1b80249bd462, State: Running, Role: LEADER
I20260619 20:48:33.638248 3075 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:33.638309 3080 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8495f9effe94961a49f1b80249bd462" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39991 } }
I20260619 20:48:33.638729 3082 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8495f9effe94961a49f1b80249bd462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8495f9effe94961a49f1b80249bd462" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39991 } } }
I20260619 20:48:33.638739 3081 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8495f9effe94961a49f1b80249bd462. Latest consensus state: current_term: 1 leader_uuid: "c8495f9effe94961a49f1b80249bd462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8495f9effe94961a49f1b80249bd462" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39991 } } }
I20260619 20:48:33.638835 3082 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:33.638839 3081 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462 [sys.catalog]: This master's current role is: LEADER
W20260619 20:48:33.640143 3096 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:33.640198 3096 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:33.640529 3087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:33.640733 3087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:33.642128 3087 catalog_manager.cc:1347] Generated new cluster ID: 679e3ba532994832b74485727ba176b4
I20260619 20:48:33.642187 3087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:33.643031 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:33.636840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:48733 (local address 127.0.0.1:39991)
0619 20:48:33.637102 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:33.637113 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:33.637158 (+ 45us) server_negotiation.cc:407] Connection header received
0619 20:48:33.637327 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:33.637337 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:33.637411 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:33.637569 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:33.637735 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:33.639441 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:33.640512 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:33.641202 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:33.641323 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:33.642329 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:33.642332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:33.642332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:33.642569 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:33.642600 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:33.642612 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:33.642703 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:33.642782 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 20:48:33.642811 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
I20260619 20:48:33.643896 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:33.648105 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:33.645187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:35853 (local address 127.0.0.1:39991)
0619 20:48:33.645221 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:33.645223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:33.645346 (+ 123us) server_negotiation.cc:407] Connection header received
0619 20:48:33.645401 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:33.645402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:33.645413 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:33.645474 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:33.645575 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:33.646061 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:33.646873 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:33.647113 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:33.647198 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:33.647824 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:33.647825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:33.647826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:33.647830 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:33.647843 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:33.647847 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:33.647899 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:33.648002 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 20:48:33.648014 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260619 20:48:33.649273 3087 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:33.650022 3087 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:33.656952 3087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8495f9effe94961a49f1b80249bd462: Generated new TSK 0
I20260619 20:48:33.657140 3087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:33.665807 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3007
W20260619 20:48:33.670660 2991 connection.cc:535] client connection to 127.0.0.1:39991 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39991): Connection reset by peer (error 104)
2026-06-19T20:48:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:33 dist-test-slave-n2td krb5kdc[3107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:33 dist-test-slave-n2td krb5kdc[3107](info): set up 2 sockets
Jun 19 20:48:33 dist-test-slave-n2td krb5kdc[3107](info): commencing operation
krb5kdc: starting...
W20260619 20:48:34.737448 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:34 dist-test-slave-n2td krb5kdc[3107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:48:34.854728 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:34.935105 3120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:34.935316 3120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:34.935338 3120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:34.936486 3120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:34.936528 3120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:34.936542 3120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:34.936561 3120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:34.936573 3120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:34.938079 3120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:34.938342 3120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:34.938599 3120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:48:34.943584 3120 server_base.cc:1034] running on GCE node
W20260619 20:48:34.943547 3128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:34.943562 3125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:34.943554 3126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:34.944010 3120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:34.944295 3120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:34.945469 3120 hybrid_clock.cc:648] HybridClock initialized: now 1781902114945464 us; error 38 us; skew 500 ppm
Jun 19 20:48:34 dist-test-slave-n2td krb5kdc[3107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:34.948046 3120 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:48:34.948490 3120 webserver.cc:460] Webserver started at http://127.0.0.1:40697/ using document root <none> and password file <none>
I20260619 20:48:34.948676 3120 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:34.948717 3120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:34.948807 3120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:34.949914 3120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "32834cf5b585457f9207d4e18430f6fc"
format_stamp: "Formatted at 2026-06-19 20:48:34 on dist-test-slave-n2td"
server_key: "4ea238869dfd8fb1cab0a310b3905b8b"
server_key_iv: "d6ff9946432d41a61f73bf70d44f52b3"
server_key_version: "encryptionkey@0"
I20260619 20:48:34.950222 3120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "32834cf5b585457f9207d4e18430f6fc"
format_stamp: "Formatted at 2026-06-19 20:48:34 on dist-test-slave-n2td"
server_key: "4ea238869dfd8fb1cab0a310b3905b8b"
server_key_iv: "d6ff9946432d41a61f73bf70d44f52b3"
server_key_version: "encryptionkey@0"
I20260619 20:48:34.951577 3120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:34.952284 3135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:34.952564 3120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:34.952708 3120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "32834cf5b585457f9207d4e18430f6fc"
format_stamp: "Formatted at 2026-06-19 20:48:34 on dist-test-slave-n2td"
server_key: "4ea238869dfd8fb1cab0a310b3905b8b"
server_key_iv: "d6ff9946432d41a61f73bf70d44f52b3"
server_key_version: "encryptionkey@0"
I20260619 20:48:34.952788 3120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:34.956382 3120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:34.958295 3120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:34.958514 3120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:34.962754 3120 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32951
I20260619 20:48:34.962782 3187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32951 every 8 connection(s)
I20260619 20:48:34.963150 3120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:34.963969 3188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:34.966185 3188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc: Bootstrap starting.
I20260619 20:48:34.966811 3188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:34.967101 3188 log.cc:824] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:34.967787 3188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc: No bootstrap required, opened a new log
I20260619 20:48:34.969321 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3120
I20260619 20:48:34.969419 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:34.969390 3188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32834cf5b585457f9207d4e18430f6fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32951 } }
I20260619 20:48:34.969597 3188 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:34.969642 14093 external_mini_cluster.cc:1428] Setting key 648812acb7d7a59be09a893a99ba71a1
I20260619 20:48:34.969643 3188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32834cf5b585457f9207d4e18430f6fc, State: Initialized, Role: FOLLOWER
I20260619 20:48:34.969799 3188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32834cf5b585457f9207d4e18430f6fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32951 } }
I20260619 20:48:34.969897 3188 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:34.969940 3188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:34.970023 3188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:34.970975 3188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32834cf5b585457f9207d4e18430f6fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32951 } }
I20260619 20:48:34.971096 3188 leader_election.cc:302] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32834cf5b585457f9207d4e18430f6fc; no voters:
I20260619 20:48:34.971256 3188 leader_election.cc:288] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:34.971309 3193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:34.971446 3193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [term 1 LEADER]: Becoming Leader. State: Replica: 32834cf5b585457f9207d4e18430f6fc, State: Running, Role: LEADER
I20260619 20:48:34.971557 3188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:34.971604 3193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32834cf5b585457f9207d4e18430f6fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32951 } }
I20260619 20:48:34.971964 3194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32834cf5b585457f9207d4e18430f6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32834cf5b585457f9207d4e18430f6fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32951 } } }
I20260619 20:48:34.972007 3195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32834cf5b585457f9207d4e18430f6fc. Latest consensus state: current_term: 1 leader_uuid: "32834cf5b585457f9207d4e18430f6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32834cf5b585457f9207d4e18430f6fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32951 } } }
I20260619 20:48:34.972057 3194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:34.972074 3195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:34.972311 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:34.970097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:45765 (local address 127.0.0.1:32951)
0619 20:48:34.970290 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:34.970300 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:34.970336 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:48:34.970562 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:34.970568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:34.970601 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:34.970761 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:34.971496 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:34.971500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:34.971500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:34.971505 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:34.971552 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:34.971567 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:34.971633 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:34.971833 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 20:48:34.971880 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260619 20:48:34.972986 3202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:34.973429 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:34.973542 3202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:34.975159 3202 catalog_manager.cc:1347] Generated new cluster ID: 50135de0899241f0bbd412a453c2c960
I20260619 20:48:34.975214 3202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:34.976114 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:34.974938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.19:38041 (local address 127.0.0.1:32951)
0619 20:48:34.975006 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:34.975008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:34.975110 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:48:34.975199 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:34.975200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:34.975205 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:34.975249 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:34.975880 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:34.975881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:34.975882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:34.975886 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:34.975900 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:34.975904 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:34.975955 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:34.976008 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 20:48:34.976024 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 20:48:34.985262 3202 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:34.985455 3202 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:34.992836 3202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32834cf5b585457f9207d4e18430f6fc: Generated new TSK 0
I20260619 20:48:34.993038 3202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:34.999954 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3120
2026-06-19T20:48:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:35 dist-test-slave-n2td krb5kdc[3219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:35 dist-test-slave-n2td krb5kdc[3219](info): set up 2 sockets
Jun 19 20:48:35 dist-test-slave-n2td krb5kdc[3219](info): commencing operation
krb5kdc: starting...
W20260619 20:48:36.037562 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:36 dist-test-slave-n2td krb5kdc[3219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:36.152535 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44527
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:44527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:36.234313 3233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:36.234534 3233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:36.234558 3233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:36.235713 3233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:36.235754 3233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:36.235769 3233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:36.235805 3233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:36.235831 3233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:36.237324 3233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:44527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:44527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:36.237574 3233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:36.237799 3233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:36.242465 3239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:36.242581 3241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:36.242456 3238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:36.242690 3233 server_base.cc:1034] running on GCE node
I20260619 20:48:36.242902 3233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:36.243191 3233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:36.244365 3233 hybrid_clock.cc:648] HybridClock initialized: now 1781902116244342 us; error 40 us; skew 500 ppm
Jun 19 20:48:36 dist-test-slave-n2td krb5kdc[3219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902116, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:36.246673 3233 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:36.247117 3233 webserver.cc:460] Webserver started at http://127.13.195.126:35109/ using document root <none> and password file <none>
I20260619 20:48:36.247334 3233 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:36.247398 3233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:36.247542 3233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:36.248565 3233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "5e6f5278828f47c6a8c4df40c320137e"
format_stamp: "Formatted at 2026-06-19 20:48:36 on dist-test-slave-n2td"
server_key: "360f77860581b1975ea7cfd3e9af2330"
server_key_iv: "f2c927d89b29875a7fc434ce20467e62"
server_key_version: "encryptionkey@0"
I20260619 20:48:36.248920 3233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "5e6f5278828f47c6a8c4df40c320137e"
format_stamp: "Formatted at 2026-06-19 20:48:36 on dist-test-slave-n2td"
server_key: "360f77860581b1975ea7cfd3e9af2330"
server_key_iv: "f2c927d89b29875a7fc434ce20467e62"
server_key_version: "encryptionkey@0"
I20260619 20:48:36.250226 3233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:36.251075 3248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:36.251291 3233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:48:36.251390 3233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "5e6f5278828f47c6a8c4df40c320137e"
format_stamp: "Formatted at 2026-06-19 20:48:36 on dist-test-slave-n2td"
server_key: "360f77860581b1975ea7cfd3e9af2330"
server_key_iv: "f2c927d89b29875a7fc434ce20467e62"
server_key_version: "encryptionkey@0"
I20260619 20:48:36.251474 3233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:36.262755 3233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:36.264501 3233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:36.264659 3233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:36.268538 3233 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:44527
I20260619 20:48:36.268612 3300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:44527 every 8 connection(s)
I20260619 20:48:36.268929 3233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:36.269841 3301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:36.272185 3301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e: Bootstrap starting.
I20260619 20:48:36.272763 3301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:36.273052 3301 log.cc:824] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:36.273739 3301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e: No bootstrap required, opened a new log
I20260619 20:48:36.275381 3301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6f5278828f47c6a8c4df40c320137e" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44527 } }
I20260619 20:48:36.275561 3301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:36.275586 3301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6f5278828f47c6a8c4df40c320137e, State: Initialized, Role: FOLLOWER
I20260619 20:48:36.275771 3301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6f5278828f47c6a8c4df40c320137e" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44527 } }
I20260619 20:48:36.275851 3301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:36.275903 3301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:36.275954 3301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:36.276785 3301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6f5278828f47c6a8c4df40c320137e" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44527 } }
I20260619 20:48:36.276897 3301 leader_election.cc:302] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e6f5278828f47c6a8c4df40c320137e; no voters:
I20260619 20:48:36.277100 3301 leader_election.cc:288] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:36.277187 3304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:36.277263 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3233
I20260619 20:48:36.277381 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:36.277383 3301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:36.277395 3304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [term 1 LEADER]: Becoming Leader. State: Replica: 5e6f5278828f47c6a8c4df40c320137e, State: Running, Role: LEADER
I20260619 20:48:36.277580 14093 external_mini_cluster.cc:1428] Setting key 1c255dac2fab9bbd748de5f9c385091a
I20260619 20:48:36.277542 3304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6f5278828f47c6a8c4df40c320137e" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44527 } }
I20260619 20:48:36.278293 3306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e6f5278828f47c6a8c4df40c320137e. Latest consensus state: current_term: 1 leader_uuid: "5e6f5278828f47c6a8c4df40c320137e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6f5278828f47c6a8c4df40c320137e" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44527 } } }
I20260619 20:48:36.278383 3306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:36.278481 3305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e6f5278828f47c6a8c4df40c320137e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6f5278828f47c6a8c4df40c320137e" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 44527 } } }
I20260619 20:48:36.278558 3305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:36.279222 3315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:36.279744 3315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:36.281812 3315 catalog_manager.cc:1347] Generated new cluster ID: 5f1262b9f4ff4c789756abd1a0fc0d2d
I20260619 20:48:36.281865 3315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:36 dist-test-slave-n2td krb5kdc[3219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:36.290532 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:36.278085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33727 (local address 127.13.195.126:44527)
0619 20:48:36.278703 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:36.278715 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:36.278846 (+ 131us) server_negotiation.cc:407] Connection header received
0619 20:48:36.279025 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:36.279032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:36.279449 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:36.279611 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:36.280260 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:36.281651 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:36.282828 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:36.283458 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:36.283564 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:36.286568 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:36.286572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:36.286573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:36.286597 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:36.288707 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:36.289183 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:36.289186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:36.289187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:36.289303 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:36.289570 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:36.289570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:36.289571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:36.289877 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:36.290040 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:36.290179 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 20:48:36.290277 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260619 20:48:36.291661 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:36.304996 3315 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:36.305619 3315 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:36.306787 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:36.294709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57971 (local address 127.13.195.126:44527)
0619 20:48:36.294752 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:36.294754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:36.298605 (+ 3851us) server_negotiation.cc:407] Connection header received
0619 20:48:36.298654 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:36.298655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:36.298666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:36.298730 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:36.299382 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:36.300694 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:36.301823 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:36.302176 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:36.302262 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:36.303800 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:36.303802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:36.303802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:36.303810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:36.305437 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:36.305900 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:36.305901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:36.305901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:36.305962 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:36.306207 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:36.306208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:36.306209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:36.306334 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:36.306495 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:36.306628 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 20:48:36.306695 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:48:36.316682 3315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e6f5278828f47c6a8c4df40c320137e: Generated new TSK 0
I20260619 20:48:36.316892 3315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:36.326054 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3233
W20260619 20:48:36.330336 3217 connection.cc:535] client connection to 127.13.195.126:44527 recv error: Network error: failed to read from TLS socket (remote: 127.13.195.126:44527): Connection reset by peer (error 104)
2026-06-19T20:48:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:36 dist-test-slave-n2td krb5kdc[3332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:36 dist-test-slave-n2td krb5kdc[3332](info): set up 2 sockets
Jun 19 20:48:36 dist-test-slave-n2td krb5kdc[3332](info): commencing operation
krb5kdc: starting...
W20260619 20:48:37.361344 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:37 dist-test-slave-n2td krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:37.477133 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:41477
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:41477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:37.557197 3345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:37.557389 3345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:37.557410 3345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:37.558583 3345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:37.558624 3345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:37.558638 3345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:37.558651 3345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:37.558706 3345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:37.560148 3345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:41477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:41477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:37.560392 3345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:37.560626 3345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:37.565410 3350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:37.565413 3353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:37.565410 3351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:37.565661 3345 server_base.cc:1034] running on GCE node
I20260619 20:48:37.566013 3345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:37.566308 3345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:37.567447 3345 hybrid_clock.cc:648] HybridClock initialized: now 1781902117567423 us; error 42 us; skew 500 ppm
Jun 19 20:48:37 dist-test-slave-n2td krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902117, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:37.573222 3345 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:37.573606 3345 webserver.cc:460] Webserver started at http://127.13.195.126:39395/ using document root <none> and password file <none>
I20260619 20:48:37.573802 3345 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:37.573843 3345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:37.573930 3345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:37.575002 3345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "16eeacd2da9540349f70c571b51f96f2"
format_stamp: "Formatted at 2026-06-19 20:48:37 on dist-test-slave-n2td"
server_key: "53d45d8629bc3aa80ca33ddeb8501bc5"
server_key_iv: "ca63b871da14f89eaae000c562e7805a"
server_key_version: "encryptionkey@0"
I20260619 20:48:37.575315 3345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "16eeacd2da9540349f70c571b51f96f2"
format_stamp: "Formatted at 2026-06-19 20:48:37 on dist-test-slave-n2td"
server_key: "53d45d8629bc3aa80ca33ddeb8501bc5"
server_key_iv: "ca63b871da14f89eaae000c562e7805a"
server_key_version: "encryptionkey@0"
I20260619 20:48:37.576532 3345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:48:37.577204 3360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:37.577378 3345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:37.577461 3345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "16eeacd2da9540349f70c571b51f96f2"
format_stamp: "Formatted at 2026-06-19 20:48:37 on dist-test-slave-n2td"
server_key: "53d45d8629bc3aa80ca33ddeb8501bc5"
server_key_iv: "ca63b871da14f89eaae000c562e7805a"
server_key_version: "encryptionkey@0"
I20260619 20:48:37.577515 3345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:37.609781 3345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:37.612361 3345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:37.612520 3345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:37.616446 3345 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:41477
I20260619 20:48:37.616475 3413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:41477 every 8 connection(s)
I20260619 20:48:37.616835 3345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:37.617707 3414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:37.620031 3414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2: Bootstrap starting.
I20260619 20:48:37.620635 3414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:37.620918 3414 log.cc:824] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:37.621603 3414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2: No bootstrap required, opened a new log
I20260619 20:48:37.622020 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3345
I20260619 20:48:37.622159 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:37.622380 14093 external_mini_cluster.cc:1428] Setting key 79fe77ac03961082268917f4927a31ef
I20260619 20:48:37.623675 3414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16eeacd2da9540349f70c571b51f96f2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 41477 } }
I20260619 20:48:37.623853 3414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:37.623893 3414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16eeacd2da9540349f70c571b51f96f2, State: Initialized, Role: FOLLOWER
I20260619 20:48:37.624053 3414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16eeacd2da9540349f70c571b51f96f2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 41477 } }
I20260619 20:48:37.624137 3414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:37.624161 3414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:37.624233 3414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:37.624771 3414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16eeacd2da9540349f70c571b51f96f2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 41477 } }
I20260619 20:48:37.624851 3414 leader_election.cc:302] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16eeacd2da9540349f70c571b51f96f2; no voters:
I20260619 20:48:37.625075 3414 leader_election.cc:288] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:37.625099 3419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:37.625208 3419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [term 1 LEADER]: Becoming Leader. State: Replica: 16eeacd2da9540349f70c571b51f96f2, State: Running, Role: LEADER
I20260619 20:48:37.625339 3414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:37.625422 3419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16eeacd2da9540349f70c571b51f96f2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 41477 } }
I20260619 20:48:37.625777 3419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16eeacd2da9540349f70c571b51f96f2. Latest consensus state: current_term: 1 leader_uuid: "16eeacd2da9540349f70c571b51f96f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16eeacd2da9540349f70c571b51f96f2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 41477 } } }
I20260619 20:48:37.625787 3420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16eeacd2da9540349f70c571b51f96f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16eeacd2da9540349f70c571b51f96f2" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 41477 } } }
I20260619 20:48:37.625869 3419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:37.625890 3420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2 [sys.catalog]: This master's current role is: LEADER
W20260619 20:48:37.627123 3434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:37.628458 3434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:37.628546 3426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:37.628651 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:37.622887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38857 (local address 127.13.195.126:41477)
0619 20:48:37.623073 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:37.623080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:37.623105 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:48:37.623647 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:37.623653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:37.623698 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:37.623807 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:37.624107 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:37.625639 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:37.626566 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:37.627179 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:37.627283 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:37.627928 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:37.627931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:37.627931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:37.628112 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:37.628166 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:37.628177 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:37.628247 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:37.628365 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 20:48:37.628407 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260619 20:48:37.628724 3426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:37.629510 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:37.631430 3426 catalog_manager.cc:1347] Generated new cluster ID: 4b288f2c5f4e43778d9099d0b34e67e9
I20260619 20:48:37.631482 3426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:37.636399 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:37.630935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43563 (local address 127.13.195.126:41477)
0619 20:48:37.630993 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:37.630994 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:37.631054 (+ 60us) server_negotiation.cc:407] Connection header received
0619 20:48:37.631160 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:37.631161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:37.631171 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:37.631240 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:37.631387 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:37.631900 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:37.635185 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:37.635410 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:37.635491 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:37.636174 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:37.636176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:37.636176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:37.636180 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:37.636194 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:37.636198 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:37.636251 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:37.636311 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:48:37.636322 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:48:37.650545 3426 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:37.651074 3426 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:37.655105 3426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16eeacd2da9540349f70c571b51f96f2: Generated new TSK 0
I20260619 20:48:37.655305 3426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:37.659896 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3345
W20260619 20:48:37.664278 3330 connection.cc:535] client connection to 127.13.195.126:41477 recv error: Network error: failed to read from TLS socket (remote: 127.13.195.126:41477): Connection reset by peer (error 104)
2026-06-19T20:48:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:37 dist-test-slave-n2td krb5kdc[3445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:37 dist-test-slave-n2td krb5kdc[3445](info): set up 2 sockets
Jun 19 20:48:37 dist-test-slave-n2td krb5kdc[3445](info): commencing operation
krb5kdc: starting...
W20260619 20:48:38.693499 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:38 dist-test-slave-n2td krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:38.811517 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36247
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:36247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:38.892921 3458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:38.893144 3458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:38.893168 3458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:38.894348 3458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:38.894392 3458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:38.894407 3458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:38.894419 3458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:38.894510 3458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:38.896025 3458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:36247
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:36247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:38.896287 3458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:38.896519 3458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:38.901310 3463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:38.901319 3464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:38.901348 3466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:38.901415 3458 server_base.cc:1034] running on GCE node
I20260619 20:48:38.901903 3458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:38.902222 3458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:38.903367 3458 hybrid_clock.cc:648] HybridClock initialized: now 1781902118903335 us; error 48 us; skew 500 ppm
Jun 19 20:48:38 dist-test-slave-n2td krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902118, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:38.905783 3458 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:38.906260 3458 webserver.cc:460] Webserver started at http://127.13.195.126:45229/ using document root <none> and password file <none>
I20260619 20:48:38.906558 3458 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:38.906628 3458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:38.906762 3458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:38.907800 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "c695846b13a3453e892acfd1534be1c8"
format_stamp: "Formatted at 2026-06-19 20:48:38 on dist-test-slave-n2td"
server_key: "daac51aa79793ebc878dd4669371f315"
server_key_iv: "250f4e4104c6f770a729658d102aa481"
server_key_version: "encryptionkey@0"
I20260619 20:48:38.908181 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "c695846b13a3453e892acfd1534be1c8"
format_stamp: "Formatted at 2026-06-19 20:48:38 on dist-test-slave-n2td"
server_key: "daac51aa79793ebc878dd4669371f315"
server_key_iv: "250f4e4104c6f770a729658d102aa481"
server_key_version: "encryptionkey@0"
I20260619 20:48:38.909494 3458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:38.910177 3473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:38.910348 3458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:38.910465 3458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "c695846b13a3453e892acfd1534be1c8"
format_stamp: "Formatted at 2026-06-19 20:48:38 on dist-test-slave-n2td"
server_key: "daac51aa79793ebc878dd4669371f315"
server_key_iv: "250f4e4104c6f770a729658d102aa481"
server_key_version: "encryptionkey@0"
I20260619 20:48:38.910547 3458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:38.913782 3458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:38.915637 3458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:38.915794 3458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:38.919826 3458 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:36247
I20260619 20:48:38.919860 3525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:36247 every 8 connection(s)
I20260619 20:48:38.920181 3458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:38.921137 3526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:38.923614 3526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8: Bootstrap starting.
I20260619 20:48:38.924183 3526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:38.924472 3526 log.cc:824] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:38.925285 3526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8: No bootstrap required, opened a new log
I20260619 20:48:38.925915 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3458
I20260619 20:48:38.926048 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:38.926225 14093 external_mini_cluster.cc:1428] Setting key f0867b8053531496ada7fe4cb95bd93f
I20260619 20:48:38.927852 3526 raft_consensus.cc:348] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c695846b13a3453e892acfd1534be1c8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36247 } }
I20260619 20:48:38.928078 3526 raft_consensus.cc:374] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:38.928191 3526 raft_consensus.cc:729] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c695846b13a3453e892acfd1534be1c8, State: Initialized, Role: FOLLOWER
I20260619 20:48:38.928364 3526 consensus_queue.cc:260] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c695846b13a3453e892acfd1534be1c8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36247 } }
I20260619 20:48:38.928469 3526 raft_consensus.cc:388] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:38.928516 3526 raft_consensus.cc:482] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:38.928578 3526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:38.929270 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:38.926772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42897 (local address 127.13.195.126:36247)
0619 20:48:38.926950 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:38.926957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:38.926981 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:48:38.927783 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:38.927790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:38.927817 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:38.927928 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:38.928819 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:38.928822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:38.928823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:38.928826 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:38.928860 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:38.928870 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:38.928925 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:38.929012 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:48:38.929043 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260619 20:48:38.929291 3526 raft_consensus.cc:504] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c695846b13a3453e892acfd1534be1c8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36247 } }
I20260619 20:48:38.929420 3526 leader_election.cc:302] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c695846b13a3453e892acfd1534be1c8; no voters:
I20260619 20:48:38.929577 3526 leader_election.cc:288] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:38.929615 3531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:38.929723 3531 raft_consensus.cc:686] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [term 1 LEADER]: Becoming Leader. State: Replica: c695846b13a3453e892acfd1534be1c8, State: Running, Role: LEADER
I20260619 20:48:38.929818 3526 sys_catalog.cc:564] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:38.929800 3531 consensus_queue.cc:237] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c695846b13a3453e892acfd1534be1c8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36247 } }
I20260619 20:48:38.930229 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:38.930186 3533 sys_catalog.cc:455] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c695846b13a3453e892acfd1534be1c8. Latest consensus state: current_term: 1 leader_uuid: "c695846b13a3453e892acfd1534be1c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c695846b13a3453e892acfd1534be1c8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36247 } } }
I20260619 20:48:38.930270 3533 sys_catalog.cc:458] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:38.930410 3532 sys_catalog.cc:455] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c695846b13a3453e892acfd1534be1c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c695846b13a3453e892acfd1534be1c8" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 36247 } } }
I20260619 20:48:38.930492 3532 sys_catalog.cc:458] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:38.931371 3535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:38.931859 3535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:38.933429 3535 catalog_manager.cc:1347] Generated new cluster ID: b21273e9086c438a9fbb0780ed8c2de8
I20260619 20:48:38.933470 3535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:38.939734 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:38.931811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45973 (local address 127.13.195.126:36247)
0619 20:48:38.931848 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:38.931851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:38.938638 (+ 6787us) server_negotiation.cc:407] Connection header received
0619 20:48:38.938720 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:38.938722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:38.938728 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:38.938783 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:38.939480 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:38.939481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:38.939482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:38.939486 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:38.939502 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:38.939506 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:38.939580 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:38.939646 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 20:48:38.939659 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:38.951159 3535 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:38.951354 3535 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:38.957119 3535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c695846b13a3453e892acfd1534be1c8: Generated new TSK 0
I20260619 20:48:38.957307 3535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:38.963249 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3458
2026-06-19T20:48:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:39 dist-test-slave-n2td krb5kdc[3558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:39 dist-test-slave-n2td krb5kdc[3558](info): set up 2 sockets
Jun 19 20:48:39 dist-test-slave-n2td krb5kdc[3558](info): commencing operation
krb5kdc: starting...
W20260619 20:48:40.005435 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:40 dist-test-slave-n2td krb5kdc[3558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:40.124904 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:37219
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:35325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:37219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:40.206346 3572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:40.206635 3572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:40.206658 3572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:40.207767 3572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:40.207801 3572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:40.207815 3572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:40.207828 3572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:40.207840 3572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:40.209232 3572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:35325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:37219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:37219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:40.209431 3572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:40.209600 3572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:40.214439 3581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:40.214494 3578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:40.214506 3579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:40.214854 3572 server_base.cc:1034] running on GCE node
I20260619 20:48:40.215081 3572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:40.215353 3572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:40.216516 3572 hybrid_clock.cc:648] HybridClock initialized: now 1781902120216483 us; error 64 us; skew 500 ppm
Jun 19 20:48:40 dist-test-slave-n2td krb5kdc[3558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902120, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:40.218995 3572 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:40.219430 3572 webserver.cc:460] Webserver started at http://127.13.195.126:40165/ using document root <none> and password file <none>
I20260619 20:48:40.219667 3572 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:40.219734 3572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:40.219889 3572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:40.220913 3572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "404f377a1c414bb692d3d73a2a299b47"
format_stamp: "Formatted at 2026-06-19 20:48:40 on dist-test-slave-n2td"
server_key: "256c1994e010fb01efa9487592d06c49"
server_key_iv: "c4c5058b073fa67a27445e6a16189118"
server_key_version: "encryptionkey@0"
I20260619 20:48:40.221316 3572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "404f377a1c414bb692d3d73a2a299b47"
format_stamp: "Formatted at 2026-06-19 20:48:40 on dist-test-slave-n2td"
server_key: "256c1994e010fb01efa9487592d06c49"
server_key_iv: "c4c5058b073fa67a27445e6a16189118"
server_key_version: "encryptionkey@0"
I20260619 20:48:40.222785 3572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:40.223644 3588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:40.223834 3572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:40.223933 3572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "404f377a1c414bb692d3d73a2a299b47"
format_stamp: "Formatted at 2026-06-19 20:48:40 on dist-test-slave-n2td"
server_key: "256c1994e010fb01efa9487592d06c49"
server_key_iv: "c4c5058b073fa67a27445e6a16189118"
server_key_version: "encryptionkey@0"
I20260619 20:48:40.224017 3572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:40.244939 3572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:40.246981 3572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:40.247159 3572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:40.251091 3572 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:37219
I20260619 20:48:40.251122 3640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:37219 every 8 connection(s)
I20260619 20:48:40.251479 3572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:40.252475 3641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:40.254839 3641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47: Bootstrap starting.
I20260619 20:48:40.255386 3641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:40.255628 3641 log.cc:824] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:40.256314 3641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47: No bootstrap required, opened a new log
I20260619 20:48:40.257963 3641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f377a1c414bb692d3d73a2a299b47" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 37219 } }
I20260619 20:48:40.258103 3641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:40.258126 3641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404f377a1c414bb692d3d73a2a299b47, State: Initialized, Role: FOLLOWER
I20260619 20:48:40.258279 3641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f377a1c414bb692d3d73a2a299b47" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 37219 } }
I20260619 20:48:40.258365 3641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:40.258389 3641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:40.258419 3641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:40.259286 3641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f377a1c414bb692d3d73a2a299b47" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 37219 } }
I20260619 20:48:40.259370 3641 leader_election.cc:302] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 404f377a1c414bb692d3d73a2a299b47; no voters:
I20260619 20:48:40.259502 3641 leader_election.cc:288] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:40.259583 3644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:40.259699 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3572
I20260619 20:48:40.259771 3641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:40.259819 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:40.259794 3644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [term 1 LEADER]: Becoming Leader. State: Replica: 404f377a1c414bb692d3d73a2a299b47, State: Running, Role: LEADER
I20260619 20:48:40.260016 14093 external_mini_cluster.cc:1428] Setting key 0f4633beca3ad12bc583625fb8fa4663
I20260619 20:48:40.259907 3644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f377a1c414bb692d3d73a2a299b47" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 37219 } }
I20260619 20:48:40.260854 3646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 404f377a1c414bb692d3d73a2a299b47. Latest consensus state: current_term: 1 leader_uuid: "404f377a1c414bb692d3d73a2a299b47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f377a1c414bb692d3d73a2a299b47" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 37219 } } }
I20260619 20:48:40.260973 3646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:40.261185 3656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:40.261377 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "404f377a1c414bb692d3d73a2a299b47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404f377a1c414bb692d3d73a2a299b47" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 37219 } } }
I20260619 20:48:40.261531 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:40.261786 3656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:40.263607 3656 catalog_manager.cc:1347] Generated new cluster ID: 149c0ae701c14cdea4fbc4f2f3a86671
I20260619 20:48:40.263662 3656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:48:40 dist-test-slave-n2td krb5kdc[3558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:40.272418 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:40.260633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45889 (local address 127.13.195.126:37219)
0619 20:48:40.260921 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:40.260931 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:40.260965 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:48:40.261175 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:40.261183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:40.261251 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:40.262311 (+ 1060us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:40.262403 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:40.263685 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:40.264819 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:40.265452 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:40.265551 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:40.268466 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:40.268471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:40.268471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:40.268485 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:40.270734 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:40.271137 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:40.271140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:40.271141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:40.271267 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:40.271478 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:40.271479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:40.271480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:40.271805 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:40.271972 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:40.272044 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 20:48:40.272187 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":58,"threads_started":1}
I20260619 20:48:40.273666 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:40.277037 3656 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:40.277691 3656 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:40.281531 3656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 404f377a1c414bb692d3d73a2a299b47: Generated new TSK 0
I20260619 20:48:40.281723 3656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:40.282347 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:40.275235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41745 (local address 127.13.195.126:37219)
0619 20:48:40.275269 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:40.275271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:40.275281 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:40.275303 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:40.275304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:40.275313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:40.275378 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:40.276001 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:40.276779 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:40.277773 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:40.278094 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:40.278166 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:40.279558 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:40.279559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:40.279559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:40.279566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:40.281119 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:40.281491 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:40.281492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:40.281492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:40.281551 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:40.281802 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:40.281803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:40.281803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:40.281925 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:40.281996 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:40.282211 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 20:48:40.282291 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:48:40.284163 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3572
2026-06-19T20:48:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:40 dist-test-slave-n2td krb5kdc[3672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:40 dist-test-slave-n2td krb5kdc[3672](info): set up 2 sockets
Jun 19 20:48:40 dist-test-slave-n2td krb5kdc[3672](info): commencing operation
krb5kdc: starting...
W20260619 20:48:41.325497 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:41 dist-test-slave-n2td krb5kdc[3672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:41.445222 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32965
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:41903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:32965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:41.525888 3685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:41.526103 3685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:41.526126 3685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:41.527313 3685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:41.527356 3685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:41.527370 3685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:41.527382 3685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:41.527395 3685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:41.528878 3685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:41903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:32965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:32965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:41.529147 3685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:41.529378 3685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:41.534053 3693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:41.534109 3690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:41.534109 3691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:41.534330 3685 server_base.cc:1034] running on GCE node
I20260619 20:48:41.534616 3685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:41.534857 3685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:41.535995 3685 hybrid_clock.cc:648] HybridClock initialized: now 1781902121535980 us; error 31 us; skew 500 ppm
Jun 19 20:48:41 dist-test-slave-n2td krb5kdc[3672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902121, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:41.538340 3685 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:41.538830 3685 webserver.cc:460] Webserver started at http://127.13.195.126:46735/ using document root <none> and password file <none>
I20260619 20:48:41.539050 3685 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:41.539131 3685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:41.539285 3685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:41.540341 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "54d28cd98f564082b6c8a7f065e68e53"
format_stamp: "Formatted at 2026-06-19 20:48:41 on dist-test-slave-n2td"
server_key: "9eddf2344a86168e7fe08fb8dcf82ea2"
server_key_iv: "f821817a50c8a374cde4fc76f2514c66"
server_key_version: "encryptionkey@0"
I20260619 20:48:41.540706 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "54d28cd98f564082b6c8a7f065e68e53"
format_stamp: "Formatted at 2026-06-19 20:48:41 on dist-test-slave-n2td"
server_key: "9eddf2344a86168e7fe08fb8dcf82ea2"
server_key_iv: "f821817a50c8a374cde4fc76f2514c66"
server_key_version: "encryptionkey@0"
I20260619 20:48:41.542011 3685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:48:41.542807 3700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:41.542977 3685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:41.543095 3685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "54d28cd98f564082b6c8a7f065e68e53"
format_stamp: "Formatted at 2026-06-19 20:48:41 on dist-test-slave-n2td"
server_key: "9eddf2344a86168e7fe08fb8dcf82ea2"
server_key_iv: "f821817a50c8a374cde4fc76f2514c66"
server_key_version: "encryptionkey@0"
I20260619 20:48:41.543174 3685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:41.566666 3685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:41.568511 3685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:41.568682 3685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:41.573104 3685 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:32965
I20260619 20:48:41.573158 3752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:32965 every 8 connection(s)
I20260619 20:48:41.573526 3685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:41.574354 3753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:41.576640 3753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53: Bootstrap starting.
I20260619 20:48:41.577214 3753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:41.577507 3753 log.cc:824] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:41.578174 3753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53: No bootstrap required, opened a new log
I20260619 20:48:41.579743 3753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d28cd98f564082b6c8a7f065e68e53" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32965 } }
I20260619 20:48:41.579878 3753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:41.579901 3753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d28cd98f564082b6c8a7f065e68e53, State: Initialized, Role: FOLLOWER
I20260619 20:48:41.580148 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3685
I20260619 20:48:41.580125 3753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d28cd98f564082b6c8a7f065e68e53" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32965 } }
I20260619 20:48:41.580228 3753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:41.580263 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:41.580278 3753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:41.580327 3753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:41.580458 14093 external_mini_cluster.cc:1428] Setting key b4f7d81e60ac3ca455caa592f6d20488
I20260619 20:48:41.581524 3753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d28cd98f564082b6c8a7f065e68e53" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32965 } }
I20260619 20:48:41.581641 3753 leader_election.cc:302] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d28cd98f564082b6c8a7f065e68e53; no voters:
I20260619 20:48:41.581791 3753 leader_election.cc:288] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:41.581837 3758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:41.582007 3758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [term 1 LEADER]: Becoming Leader. State: Replica: 54d28cd98f564082b6c8a7f065e68e53, State: Running, Role: LEADER
I20260619 20:48:41.582110 3753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:41.582131 3758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d28cd98f564082b6c8a7f065e68e53" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32965 } }
I20260619 20:48:41.582517 3759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54d28cd98f564082b6c8a7f065e68e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d28cd98f564082b6c8a7f065e68e53" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32965 } } }
I20260619 20:48:41.582506 3758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d28cd98f564082b6c8a7f065e68e53. Latest consensus state: current_term: 1 leader_uuid: "54d28cd98f564082b6c8a7f065e68e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d28cd98f564082b6c8a7f065e68e53" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 32965 } } }
I20260619 20:48:41.582669 3759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:41.582693 3758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:41.583074 3765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:41.583516 3765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:41.585664 3765 catalog_manager.cc:1347] Generated new cluster ID: d774fa768e934722810d6478c606ceda
I20260619 20:48:41.585721 3765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:41.586647 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:41.580974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40981 (local address 127.13.195.126:32965)
0619 20:48:41.581199 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:41.581211 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:41.581251 (+ 40us) server_negotiation.cc:407] Connection header received
0619 20:48:41.581368 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:41.581374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:41.581439 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:41.581593 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:41.581880 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:41.583565 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:41.584629 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:41.585041 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:41.585189 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:41.585934 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:41.585937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:41.585938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:41.586121 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:41.586157 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:41.586167 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:41.586239 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:41.586320 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 20:48:41.586382 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
I20260619 20:48:41.587417 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:41.598294 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:41.588792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56705 (local address 127.13.195.126:32965)
0619 20:48:41.588830 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:41.588831 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:41.594843 (+ 6012us) server_negotiation.cc:407] Connection header received
0619 20:48:41.594923 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:41.594924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:41.594945 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:41.595011 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:41.595158 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:41.595663 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:41.596786 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:41.597072 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:41.597107 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:41.597970 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:41.597971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:41.597972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:41.597977 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:41.597994 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:41.598000 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:41.598052 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:41.598152 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 20:48:41.598186 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:48:41.603570 3765 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:41.604355 3765 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:41.618162 3765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d28cd98f564082b6c8a7f065e68e53: Generated new TSK 0
I20260619 20:48:41.618393 3765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:41.626577 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3685
2026-06-19T20:48:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:41 dist-test-slave-n2td krb5kdc[3784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:41 dist-test-slave-n2td krb5kdc[3784](info): set up 2 sockets
Jun 19 20:48:41 dist-test-slave-n2td krb5kdc[3784](info): commencing operation
krb5kdc: starting...
W20260619 20:48:42.658077 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:42 dist-test-slave-n2td krb5kdc[3784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:42.777235 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:39365
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:39365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:42.858867 3797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:42.859114 3797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:42.859140 3797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:42.860257 3797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:42.860296 3797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:42.860311 3797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:42.860324 3797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:42.860337 3797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:42.861765 3797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:39365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:39365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:42.861954 3797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:42.862120 3797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:42.867030 3805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:42.867098 3797 server_base.cc:1034] running on GCE node
W20260619 20:48:42.867048 3803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:42.867055 3802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:42.867504 3797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:42.867790 3797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:42.868983 3797 hybrid_clock.cc:648] HybridClock initialized: now 1781902122868967 us; error 31 us; skew 500 ppm
Jun 19 20:48:42 dist-test-slave-n2td krb5kdc[3784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902122, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:42.874379 3797 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:42.874997 3797 webserver.cc:460] Webserver started at http://127.13.195.126:38261/ using document root <none> and password file <none>
I20260619 20:48:42.875213 3797 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:42.875278 3797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:42.875489 3797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:42.876614 3797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "d02fcde118474828ae45b5f126afc392"
format_stamp: "Formatted at 2026-06-19 20:48:42 on dist-test-slave-n2td"
server_key: "7d820dc323865e241a1d2cc566f1b50e"
server_key_iv: "6d53ed1cc1ad30936adeb8924dff43e9"
server_key_version: "encryptionkey@0"
I20260619 20:48:42.876979 3797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "d02fcde118474828ae45b5f126afc392"
format_stamp: "Formatted at 2026-06-19 20:48:42 on dist-test-slave-n2td"
server_key: "7d820dc323865e241a1d2cc566f1b50e"
server_key_iv: "6d53ed1cc1ad30936adeb8924dff43e9"
server_key_version: "encryptionkey@0"
I20260619 20:48:42.878304 3797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:42.879174 3812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:42.879374 3797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:42.879514 3797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "d02fcde118474828ae45b5f126afc392"
format_stamp: "Formatted at 2026-06-19 20:48:42 on dist-test-slave-n2td"
server_key: "7d820dc323865e241a1d2cc566f1b50e"
server_key_iv: "6d53ed1cc1ad30936adeb8924dff43e9"
server_key_version: "encryptionkey@0"
I20260619 20:48:42.879603 3797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:42.882905 3797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:42.884745 3797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:42.884899 3797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:42.888881 3797 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:39365
I20260619 20:48:42.888913 3864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:39365 every 8 connection(s)
I20260619 20:48:42.889262 3797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:42.890172 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:42.892144 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3797
I20260619 20:48:42.892271 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:42.892462 14093 external_mini_cluster.cc:1428] Setting key 57a827e909ac740e303706ef4cdb9f24
I20260619 20:48:42.892753 3865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392: Bootstrap starting.
I20260619 20:48:42.893715 3865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:42.894057 3865 log.cc:824] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:42.894918 3865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392: No bootstrap required, opened a new log
I20260619 20:48:42.895470 3868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:42.893037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34001 (local address 127.13.195.126:39365)
0619 20:48:42.893206 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:42.893216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:42.893248 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:48:42.893885 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:42.893897 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:42.893936 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:42.894110 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:42.895029 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:42.895033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:42.895033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:42.895038 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:42.895090 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:42.895103 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:42.895194 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:42.895231 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 20:48:42.895271 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260619 20:48:42.896734 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:42.897538 3865 raft_consensus.cc:348] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02fcde118474828ae45b5f126afc392" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39365 } }
I20260619 20:48:42.897729 3865 raft_consensus.cc:374] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:48:42.897763 3865 raft_consensus.cc:729] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02fcde118474828ae45b5f126afc392, State: Initialized, Role: FOLLOWER
I20260619 20:48:42.897888 3865 consensus_queue.cc:260] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02fcde118474828ae45b5f126afc392" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39365 } }
I20260619 20:48:42.897993 3865 raft_consensus.cc:388] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:48:42.898046 3865 raft_consensus.cc:482] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:48:42.898103 3865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:42.898818 3865 raft_consensus.cc:504] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02fcde118474828ae45b5f126afc392" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39365 } }
I20260619 20:48:42.898908 3865 leader_election.cc:302] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d02fcde118474828ae45b5f126afc392; no voters:
I20260619 20:48:42.899096 3865 leader_election.cc:288] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:48:42.899130 3875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:42.899348 3875 raft_consensus.cc:686] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [term 1 LEADER]: Becoming Leader. State: Replica: d02fcde118474828ae45b5f126afc392, State: Running, Role: LEADER
I20260619 20:48:42.899358 3865 sys_catalog.cc:564] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:42.899502 3875 consensus_queue.cc:237] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02fcde118474828ae45b5f126afc392" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39365 } }
I20260619 20:48:42.899717 3868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:42.898393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51795 (local address 127.13.195.126:39365)
0619 20:48:42.898453 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:42.898456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:42.898605 (+ 149us) server_negotiation.cc:407] Connection header received
0619 20:48:42.898682 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:42.898683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:42.898688 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:42.898745 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:42.899402 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:42.899403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:42.899403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:48:42.899407 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:42.899422 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:48:42.899427 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:42.899473 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:42.899650 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 20:48:42.899663 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:48:42.900167 3877 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d02fcde118474828ae45b5f126afc392. Latest consensus state: current_term: 1 leader_uuid: "d02fcde118474828ae45b5f126afc392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02fcde118474828ae45b5f126afc392" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39365 } } }
I20260619 20:48:42.900156 3876 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d02fcde118474828ae45b5f126afc392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02fcde118474828ae45b5f126afc392" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 39365 } } }
I20260619 20:48:42.900251 3877 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:42.900255 3876 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:42.900466 3884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:42.901028 3884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:42.902415 3884 catalog_manager.cc:1347] Generated new cluster ID: c3781377cd2d4a1881f5f72eab1e6926
I20260619 20:48:42.902486 3884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:42.907213 3884 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:42.907387 3884 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:42.911859 3884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d02fcde118474828ae45b5f126afc392: Generated new TSK 0
I20260619 20:48:42.912062 3884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:48:42.918196 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3797
2026-06-19T20:48:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1291 ms)
[----------] 18 tests from AuthTokenIssuingTest (23833 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:42 dist-test-slave-n2td krb5kdc[3896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:42 dist-test-slave-n2td krb5kdc[3896](info): set up 2 sockets
Jun 19 20:48:42 dist-test-slave-n2td krb5kdc[3896](info): commencing operation
krb5kdc: starting...
W20260619 20:48:44.001971 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:44.121243 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:35415
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:35415,127.13.195.125:33595,127.13.195.124:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:44.201789 3909 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:48:44.201999 3909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:44.202018 3909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:44.202037 3909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:44.203168 3909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:44.203207 3909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:44.203222 3909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:44.203233 3909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:44.203245 3909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:44.204653 3909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:35415,127.13.195.125:33595,127.13.195.124:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:35415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:44.204840 3909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:44.205013 3909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:44.209861 3917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:44.209861 3915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:44.209918 3914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:44.210134 3909 server_base.cc:1034] running on GCE node
I20260619 20:48:44.210386 3909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:44.210705 3909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:44.211853 3909 hybrid_clock.cc:648] HybridClock initialized: now 1781902124211831 us; error 46 us; skew 500 ppm
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:44.214385 3909 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:44.214876 3909 webserver.cc:460] Webserver started at http://127.13.195.126:38261/ using document root <none> and password file <none>
I20260619 20:48:44.215126 3909 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:44.215193 3909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:44.215345 3909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:44.216392 3909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "cce28976f0e640348fdb967365ae338c"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "67bcf02074cb78536c3b09e3c6459761"
server_key_iv: "049f1caf07867e6b3782922a4cbe68d6"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.216753 3909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "cce28976f0e640348fdb967365ae338c"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "67bcf02074cb78536c3b09e3c6459761"
server_key_iv: "049f1caf07867e6b3782922a4cbe68d6"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.218170 3909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:48:44.219008 3924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:44.219206 3909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:48:44.219303 3909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "cce28976f0e640348fdb967365ae338c"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "67bcf02074cb78536c3b09e3c6459761"
server_key_iv: "049f1caf07867e6b3782922a4cbe68d6"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.219385 3909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:44.229975 3909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:44.232384 3909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:44.232551 3909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:44.237036 3909 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:35415
I20260619 20:48:44.237087 3976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:35415 every 8 connection(s)
I20260619 20:48:44.237445 3909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:44.238368 3977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:44.240139 3977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:44.246263 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3909
I20260619 20:48:44.246380 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:44.246608 14093 external_mini_cluster.cc:1428] Setting key 4d96da0a5ee15279461123c9ec6fbd4b
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:44.250651 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.240903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.13.195.126:35415)
0619 20:48:44.241091 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.241096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.241163 (+ 67us) server_negotiation.cc:407] Connection header received
0619 20:48:44.241867 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.241870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.241889 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.241994 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.242649 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.243534 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.244352 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.244724 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.244819 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.247189 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.247198 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.247198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.247206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.248895 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.249332 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.249334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.249334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.249407 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.249682 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.249683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.249683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.249888 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.249951 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.250225 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.250391 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260619 20:48:44.250655 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.240863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:35415 (local address 127.0.0.1:39398)
0619 20:48:44.241033 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.241082 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.241679 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.242022 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.242024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.242279 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.242477 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.242483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.243624 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.243625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.244278 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.244280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.244810 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.244810 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.244871 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.245427 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.245436 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.247095 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.248980 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.248986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.248988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.249252 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.249497 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.249498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.249499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.249619 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.249960 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.249967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.250043 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.250148 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.250335 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260619 20:48:44.251688 3977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:44.252066 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.251927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:52988)
0619 20:48:44.251982 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.252037 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:48:44.252254 3928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
I20260619 20:48:44.252705 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.252587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:52990)
0619 20:48:44.252645 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.252679 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 20:48:44.252897 3977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:33595: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.195.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab.
Entry for principal kudu/127.13.195.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab.
I20260619 20:48:44.269037 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:33595
--webserver_interface=127.13.195.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab
--principal=kudu/127.13.195.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:35415,127.13.195.125:33595,127.13.195.124:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:48:44.302153 3977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } attempt: 1
I20260619 20:48:44.302695 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.302537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:53004)
0619 20:48:44.302612 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.302658 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 20:48:44.303076 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.302989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:53020)
0619 20:48:44.303033 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.303058 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 20:48:44.303184 3977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:33595: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
W20260619 20:48:44.353506 3984 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:48:44.353718 3984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:44.353735 3984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:44.353755 3984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:44.354920 3984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:44.354967 3984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:44.354982 3984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:44.354995 3984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:44.355007 3984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:44.356400 3984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:35415,127.13.195.125:33595,127.13.195.124:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab
--principal=kudu/127.13.195.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:33595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.195.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:44.356594 3984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:44.356763 3984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:44.361651 3992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:44.361647 3989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:44.361735 3990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:44.361917 3984 server_base.cc:1034] running on GCE node
I20260619 20:48:44.362211 3984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:44.362555 3984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:44.363744 3984 hybrid_clock.cc:648] HybridClock initialized: now 1781902124363726 us; error 26 us; skew 500 ppm
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:44.366302 3984 init.cc:377] Logged in from keytab as kudu/127.13.195.125@KRBTEST.COM (short username kudu)
I20260619 20:48:44.366773 3984 webserver.cc:460] Webserver started at http://127.13.195.125:40243/ using document root <none> and password file <none>
I20260619 20:48:44.366968 3984 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:44.367012 3984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:44.367102 3984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:44.368216 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data/instance:
uuid: "f510183729d64f0ca51b7d7f78454968"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "21e32ab59b331f08246a2e09f70833cb"
server_key_iv: "724f3cb1ddd4580757f31d4dd41aae7d"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.368559 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/wal/instance:
uuid: "f510183729d64f0ca51b7d7f78454968"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "21e32ab59b331f08246a2e09f70833cb"
server_key_iv: "724f3cb1ddd4580757f31d4dd41aae7d"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.369884 3984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:44.370661 3999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:44.370814 3984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:48:44.370920 3984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/wal
uuid: "f510183729d64f0ca51b7d7f78454968"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "21e32ab59b331f08246a2e09f70833cb"
server_key_iv: "724f3cb1ddd4580757f31d4dd41aae7d"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.370975 3984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:44.371315 3977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } attempt: 2
I20260619 20:48:44.371747 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.371625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:53030)
0619 20:48:44.371683 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.371719 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:48:44.372084 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.372028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:53034)
0619 20:48:44.372050 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.372071 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 20:48:44.372182 3977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:33595: Network error: Client connection negotiation failed: client connection to 127.13.195.125:33595: connect: Connection refused (error 111)
I20260619 20:48:44.383507 3984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:44.385778 3984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:44.385928 3984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:44.389850 3984 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.125:33595
I20260619 20:48:44.389922 4052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.125:33595 every 8 connection(s)
I20260619 20:48:44.390298 3984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
I20260619 20:48:44.391201 4053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:44.392735 4053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:44.393743 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 3984
I20260619 20:48:44.393872 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-1/wal/instance
I20260619 20:48:44.394054 14093 external_mini_cluster.cc:1428] Setting key 0bc9009fb11935220e400423dd2219e1
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.124@KRBTEST.COM; defaulting to no policy
I20260619 20:48:44.402868 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.393468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.13.195.126:35415)
0619 20:48:44.393520 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.393522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.393833 (+ 311us) server_negotiation.cc:407] Connection header received
0619 20:48:44.394520 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.394522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.394535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.394591 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.395419 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.395930 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.397175 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.397520 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.397599 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.399774 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.399775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.399776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.399783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.401426 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.401811 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.401812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.401813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.401877 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.402113 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.402113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.402114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.402257 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.402339 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.402695 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.402801 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:48:44.402946 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.393435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:35415 (local address 127.0.0.1:39414)
0619 20:48:44.393655 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.393708 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.394412 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.394677 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.394681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.395051 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.395331 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.395337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.396024 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.396025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.397083 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.397086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.397606 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.397607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.397718 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.398310 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.398318 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.399679 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.401509 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.401515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.401516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.401729 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.401948 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.401948 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.401949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.402048 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.402351 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.402357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.402540 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.402629 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.402752 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260619 20:48:44.403483 4053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.124@KRBTEST.COM" created.
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:48:44.411340 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.403757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:53036)
0619 20:48:44.403806 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.403817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.403858 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.404217 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.404218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.404366 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.404447 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.404448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.405427 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.405428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.405920 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.405921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.406668 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.406669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.406731 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.407183 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.407187 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.408453 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.410221 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.410222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.410222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.410388 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.410646 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.410647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.410647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.410701 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.411005 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.411006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.411045 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.411125 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.411226 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:48:44.411376 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.403893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.13.195.125:33595)
0619 20:48:44.404048 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.404052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.404067 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:48:44.404083 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.404085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.404094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.404180 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.404534 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.405343 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.406023 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.406574 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.406671 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.408552 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.408560 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.408561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.408567 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.410106 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.410497 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.410500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.410501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.410560 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.410772 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.410773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.410773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.410924 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.411016 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.411147 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.411276 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260619 20:48:44.412089 4053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:44.412456 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.412333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40344)
0619 20:48:44.412381 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.412429 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:48:44.412593 4000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
I20260619 20:48:44.413003 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.412908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40348)
0619 20:48:44.412959 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.412982 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 20:48:44.413107 4053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:36433: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab.
Entry for principal kudu/127.13.195.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab.
I20260619 20:48:44.420984 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:36433
--webserver_interface=127.13.195.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab
--principal=kudu/127.13.195.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:33231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:35415,127.13.195.125:33595,127.13.195.124:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:48:44.462356 4053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } attempt: 1
I20260619 20:48:44.463039 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.462872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40356)
0619 20:48:44.462961 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.463008 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 20:48:44.463439 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.463330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40362)
0619 20:48:44.463380 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.463416 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:48:44.463317 3977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } attempt: 3
W20260619 20:48:44.463546 4053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:36433: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:48:44.470950 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.463696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:53052)
0619 20:48:44.463745 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.463757 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.463819 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.463970 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.463970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.464172 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.464260 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.464262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.464985 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.464986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.465818 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.465821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.466269 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.466270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.466347 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.466808 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.466814 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.468197 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.469990 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.469991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.469991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.470156 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.470353 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.470353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.470353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.470405 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.470714 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.470714 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.470755 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.470811 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.470883 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 20:48:44.471014 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.463716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.13.195.125:33595)
0619 20:48:44.463765 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.463767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.463814 (+ 47us) server_negotiation.cc:407] Connection header received
0619 20:48:44.463885 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.463885 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.463897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.463953 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.464360 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.464898 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.465920 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.466168 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.466232 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.468281 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.468282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.468282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.468289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.469901 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.470226 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.470227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.470227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.470288 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.470512 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.470513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.470514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.470651 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.470720 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.470854 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.470932 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:48:44.471467 3977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:44.471784 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.471692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40374)
0619 20:48:44.471727 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.471760 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:48:44.472138 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.472079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40382)
0619 20:48:44.472101 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.472123 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 20:48:44.472247 3977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:36433: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
I20260619 20:48:44.503418 3977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } attempt: 1
I20260619 20:48:44.503955 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.503804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40388)
0619 20:48:44.503869 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.503917 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 20:48:44.504357 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.504255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40402)
0619 20:48:44.504306 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.504334 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 20:48:44.504498 3977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:36433: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
W20260619 20:48:44.506780 4060 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:48:44.507004 4060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:44.507045 4060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:44.507086 4060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:44.508286 4060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:44.508351 4060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:44.508378 4060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:44.508405 4060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:44.508431 4060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:44.509907 4060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:33231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:35415,127.13.195.125:33595,127.13.195.124:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab
--principal=kudu/127.13.195.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:36433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.195.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:44.510183 4060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:44.510419 4060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:44.515231 4066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:44.515272 4065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:44.515447 4060 server_base.cc:1034] running on GCE node
W20260619 20:48:44.515231 4068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:44.515733 4060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:44.516065 4060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:44.517256 4060 hybrid_clock.cc:648] HybridClock initialized: now 1781902124517228 us; error 36 us; skew 500 ppm
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:44.519937 4060 init.cc:377] Logged in from keytab as kudu/127.13.195.124@KRBTEST.COM (short username kudu)
I20260619 20:48:44.520357 4060 webserver.cc:460] Webserver started at http://127.13.195.124:41823/ using document root <none> and password file <none>
I20260619 20:48:44.520705 4060 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:44.520802 4060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:44.520965 4060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:44.522176 4060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data/instance:
uuid: "6ee4c2c4fe044092ac42962ca3a58094"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "6a5d46809a7c0a685abc468e5e8c13ae"
server_key_iv: "205f9d92e8512028ef73e4adb7ff0229"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.522598 4060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/wal/instance:
uuid: "6ee4c2c4fe044092ac42962ca3a58094"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "6a5d46809a7c0a685abc468e5e8c13ae"
server_key_iv: "205f9d92e8512028ef73e4adb7ff0229"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.523933 4060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:44.524681 4075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:44.524853 4060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:44.524957 4060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/wal
uuid: "6ee4c2c4fe044092ac42962ca3a58094"
format_stamp: "Formatted at 2026-06-19 20:48:44 on dist-test-slave-n2td"
server_key: "6a5d46809a7c0a685abc468e5e8c13ae"
server_key_iv: "205f9d92e8512028ef73e4adb7ff0229"
server_key_version: "encryptionkey@0"
I20260619 20:48:44.525032 4060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:44.531682 4053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } attempt: 2
I20260619 20:48:44.532115 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.531990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40410)
0619 20:48:44.532045 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.532085 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:48:44.532477 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.532396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40426)
0619 20:48:44.532436 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.532460 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 20:48:44.532599 4053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:36433: Network error: Client connection negotiation failed: client connection to 127.13.195.124:36433: connect: Connection refused (error 111)
I20260619 20:48:44.548111 4060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:44.550580 4060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:44.550750 4060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:44.554623 4060 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.124:36433
I20260619 20:48:44.554682 4127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.124:36433 every 8 connection(s)
I20260619 20:48:44.555042 4060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
I20260619 20:48:44.555914 4128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:44.556058 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 4060
I20260619 20:48:44.556205 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901857247738-14093-0/minicluster-data/master-2/wal/instance
I20260619 20:48:44.556389 14093 external_mini_cluster.cc:1428] Setting key 40776caab056204270966ca474a63984
I20260619 20:48:44.558609 4128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:48:44.567698 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.559358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.13.195.126:35415)
0619 20:48:44.559424 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.559426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.559574 (+ 148us) server_negotiation.cc:407] Connection header received
0619 20:48:44.559633 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.559641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.559650 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.559699 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.560243 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.560971 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.562042 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.562343 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.562402 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.564569 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.564570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.564571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.564577 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.566146 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.566740 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.566741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.566741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.566791 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.567060 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.567061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.567061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.567163 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.567214 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.567528 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.567613 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:48:44.567878 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.559297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:35415 (local address 127.0.0.1:39422)
0619 20:48:44.559463 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.559503 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.559574 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.559734 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.559736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.560065 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.560154 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.560159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.561071 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.561072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.561952 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.561954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.562456 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.562457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.562540 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.562987 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.562995 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.564456 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.566249 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.566255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.566258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.566661 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.566855 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.566856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.566857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.566981 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.567229 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.567233 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.567392 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.567475 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.567617 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260619 20:48:44.568552 4128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:44.568629 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.556921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.13.195.124:36433)
0619 20:48:44.557132 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.557141 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.557173 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:48:44.557787 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.557797 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.557865 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.558034 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.558783 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.560159 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.561202 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.561851 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.561968 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.565178 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.565186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.565186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.565193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.567234 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.567620 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.567623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.567623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.567679 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.567907 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.567908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.567908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.568083 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.568258 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.568411 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.568494 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:44.575498 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.568879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.13.195.125:33595)
0619 20:48:44.568922 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.568924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.568932 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:44.568978 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.568979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.568989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.569048 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.569375 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.569933 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.570648 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.570854 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.570904 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.572887 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.572888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.572889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.572895 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.574359 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.574780 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.574781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.574782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.574842 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.575061 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.575061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.575062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.575169 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.575223 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.575380 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.575440 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:44.575551 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.568816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:33595 (local address 127.0.0.1:53056)
0619 20:48:44.568868 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.568878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.568913 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.569059 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.569060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.569205 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.569251 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.569252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.570007 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.570008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.570562 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.570564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.570924 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.570925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.571001 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.571376 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.571380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.572811 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.574416 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.574417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.574417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.574656 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.574927 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.574928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.574928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.574992 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.575236 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.575237 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.575278 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.575355 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.575438 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:48:44.575899 4128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:44.576917 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.569374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.13.195.126:35415)
0619 20:48:44.569412 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.569414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.569420 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:48:44.569439 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.569439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.569447 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.569499 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.569916 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.570403 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.571336 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.571553 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.571605 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.573877 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.573878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.573878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.573885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.575728 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.576169 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.576170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.576170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.576226 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.576441 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.576442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.576442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.576544 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.576725 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.576761 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.576831 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:44.579646 3977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } attempt: 2
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:48:44.582693 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.576146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40448)
0619 20:48:44.576208 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.576217 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.576246 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.576379 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.576379 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.576516 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.576543 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.576544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.577400 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.577400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.577913 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.577914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.578297 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.578297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.578344 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.578758 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.578763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.580009 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.581643 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.581644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.581644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.581829 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.582023 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.582024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.582024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.582089 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.582364 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.582365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.582404 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.582499 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.582582 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:48:44.582945 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.576180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40448 (local address 127.13.195.124:36433)
0619 20:48:44.576236 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.576238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.576245 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:48:44.576308 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.576309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.576316 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.576370 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.576635 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.577307 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.578006 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.578214 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.578295 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.580098 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.580099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.580099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.580105 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.581541 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.581906 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.581907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.581907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.581953 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.582166 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.582167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.582167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.582291 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.582358 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.582803 (+ 445us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.582870 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:48:44.584493 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: Bootstrap starting.
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:48:44.585208 4128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:44.585565 4128 log.cc:824] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:44.585884 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.577507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.13.195.125:33595)
0619 20:48:44.577544 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.577546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.577553 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:48:44.577574 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.577575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.577584 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.577639 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.578074 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.578617 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.579485 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.580081 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.580145 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.582940 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.582941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.582942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.582948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.584534 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.584927 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.584928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.584928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.584977 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.585187 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.585188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.585188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.585331 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.585460 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.585712 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.585785 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:44.586408 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: No bootstrap required, opened a new log
I20260619 20:48:44.587836 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:44.588330 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.579874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40462)
0619 20:48:44.579921 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.579930 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.579965 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.580237 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.580238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.580386 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.580429 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.580430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.581855 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.581856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.582622 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.582624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.583327 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.583328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.583382 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.583737 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.583742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.585124 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.587217 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.587218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.587218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.587396 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.587629 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.587630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.587630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.587685 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.587996 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.587997 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.588043 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.588124 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.588235 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:48:44.588366 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.579930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.13.195.124:36433)
0619 20:48:44.580094 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.580096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.580103 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:48:44.580121 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.580122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.580132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.580484 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.580518 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.581790 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.582702 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.583161 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.583515 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.585207 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.585208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.585209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.585215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.587133 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.587486 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.587487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.587488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.587571 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.587754 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.587755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.587755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.587906 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.587983 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.588170 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.588243 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":120,"threads_started":1}
I20260619 20:48:44.588495 4128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.588639 4128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee4c2c4fe044092ac42962ca3a58094, State: Initialized, Role: FOLLOWER
I20260619 20:48:44.588713 4128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.589016 4128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:44.588971 4136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.589066 4136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:44.589816 3977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c: Bootstrap starting.
I20260619 20:48:44.589089 3951 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cce28976f0e640348fdb967365ae338c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39438
I20260619 20:48:44.590660 3977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:44.590983 3977 log.cc:824] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c: Log is configured to *not* fsync() on all Append() calls
W20260619 20:48:44.591176 4147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:44.591229 4147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:44.591841 3977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c: No bootstrap required, opened a new log
I20260619 20:48:44.593155 3977 raft_consensus.cc:348] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.593291 3977 raft_consensus.cc:729] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce28976f0e640348fdb967365ae338c, State: Initialized, Role: FOLLOWER
I20260619 20:48:44.593369 3977 consensus_queue.cc:260] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.593660 3977 sys_catalog.cc:564] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:44.591536 4060 master_runner.cc:186] Error getting master registration for 127.13.195.126:35415: OK, instance_id { permanent_uuid: "cce28976f0e640348fdb967365ae338c" instance_seqno: 1781902124235677 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:48:44.593794 4060 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:48:44.593650 4150 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.593972 4150 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:44.593716 3951 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 20:48:44.594580 3951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:44.595360 3951 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.595613 3951 leader_election.cc:288] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [CANDIDATE]: Term 1 election: Requested vote from peers f510183729d64f0ca51b7d7f78454968 (127.13.195.125:33595), 6ee4c2c4fe044092ac42962ca3a58094 (127.13.195.124:36433)
I20260619 20:48:44.596060 4102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cce28976f0e640348fdb967365ae338c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6ee4c2c4fe044092ac42962ca3a58094"
W20260619 20:48:44.596105 4161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:44.596153 4161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:44.596170 4102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:44.597150 4102 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce28976f0e640348fdb967365ae338c in term 1.
I20260619 20:48:44.597400 3925 leader_election.cc:302] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ee4c2c4fe044092ac42962ca3a58094, cce28976f0e640348fdb967365ae338c; no voters:
I20260619 20:48:44.597662 4150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:44.597818 4150 raft_consensus.cc:686] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [term 1 LEADER]: Becoming Leader. State: Replica: cce28976f0e640348fdb967365ae338c, State: Running, Role: LEADER
I20260619 20:48:44.597908 4150 consensus_queue.cc:237] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.599277 3909 master_runner.cc:186] Error getting master registration for 127.13.195.125:33595: OK, instance_id { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" instance_seqno: 1781902124388746 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:48:44.601697 3909 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:48:44.605779 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.596910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.13.195.124:36433)
0619 20:48:44.596960 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.596963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.597574 (+ 611us) server_negotiation.cc:407] Connection header received
0619 20:48:44.597678 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.597679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.597689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.597734 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.598664 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.599420 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.600459 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.600705 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.600902 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.602633 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.602634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.602634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.602643 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.604467 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.604869 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.604870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.604871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.604935 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.605182 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.605183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.605183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.605314 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.605433 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.605573 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.605667 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:48:44.606448 4169 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce28976f0e640348fdb967365ae338c. Latest consensus state: current_term: 1 leader_uuid: "cce28976f0e640348fdb967365ae338c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.606657 4169 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:44.615491 4172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:44.615784 4172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:44.615916 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.598109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.13.195.125:33595)
0619 20:48:44.598150 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.598152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.606149 (+ 7997us) server_negotiation.cc:407] Connection header received
0619 20:48:44.606201 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.606202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.606216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.606294 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.607187 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.607924 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.609075 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.609400 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.609444 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.611190 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.611192 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.611192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.611203 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.613030 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.614813 (+ 1783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.614814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.614815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.614888 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.615101 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.615102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.615102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.615266 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.615384 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.615690 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.615788 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:48:44.616671 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.596960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.13.195.126:35415)
0619 20:48:44.597028 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.597030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.605882 (+ 8852us) server_negotiation.cc:407] Connection header received
0619 20:48:44.605975 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.605976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.605988 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.606042 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.607139 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.607910 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.609060 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.609348 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.609405 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.612854 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.612856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.612856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.612864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.614672 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.615670 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.615672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.615672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.615737 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.615976 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.615977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.615977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.616125 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.616229 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.616467 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.616553 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:48:44.623757 4053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } attempt: 3
I20260619 20:48:44.628247 4102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [term 1 FOLLOWER]: Refusing update from remote peer cce28976f0e640348fdb967365ae338c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
Jun 19 20:48:44 dist-test-slave-n2td krb5kdc[3896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902124, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:48:44.628649 4169 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:44.630103 4136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce28976f0e640348fdb967365ae338c. Latest consensus state: current_term: 1 leader_uuid: "cce28976f0e640348fdb967365ae338c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.630191 4136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:44.633159 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.624123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:36433 (local address 127.0.0.1:40484)
0619 20:48:44.624175 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:44.624187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:44.624230 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:44.624408 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:44.624409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:44.624576 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:44.624620 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.624621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.625517 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.625518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.626045 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:44.626047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.626567 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.626568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:44.626627 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.627168 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:44.627174 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:44.629038 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:44.631935 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.631936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.631937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.632128 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.632396 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:44.632397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:44.632397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:44.632492 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:44.632789 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:44.632790 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:44.632853 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 20:48:44.632941 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:48:44.633052 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:48:44.633586 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:44.624228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.13.195.124:36433)
0619 20:48:44.624274 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.624276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.624284 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:44.624308 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.624309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.624319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.624379 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.624708 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.625445 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.626155 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:44.626480 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:44.626573 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:44.629153 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:44.629155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:44.629155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:44.629163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:44.631861 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.632228 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.632229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.632229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.632292 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:44.632560 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:44.632561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:44.632561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:44.632722 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:44.632792 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:44.632993 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 20:48:44.633122 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:48:44.635192 4053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: Bootstrap starting.
I20260619 20:48:44.635991 4053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:44.636334 4053 log.cc:824] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:44.637418 4053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: No bootstrap required, opened a new log
I20260619 20:48:44.639861 4053 raft_consensus.cc:348] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.640020 4053 raft_consensus.cc:729] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f510183729d64f0ca51b7d7f78454968, State: Initialized, Role: FOLLOWER
I20260619 20:48:44.640182 4053 consensus_queue.cc:260] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } }
I20260619 20:48:44.640595 4053 sys_catalog.cc:564] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:44.640686 4026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cce28976f0e640348fdb967365ae338c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f510183729d64f0ca51b7d7f78454968"
I20260619 20:48:44.640651 4025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:44.642213 4025 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [term 1 FOLLOWER]: Refusing update from remote peer cce28976f0e640348fdb967365ae338c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:44.642772 4169 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:44.643404 4169 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cce28976f0e640348fdb967365ae338c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.643494 4169 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:44.643573 4173 mvcc.cc:205] Tried to move back new op lower bound from 7298671102471090176 to 7298671102353764352. Current Snapshot: MvccSnapshot[applied={T|T < 7298671102471090176}]
W20260619 20:48:44.643750 4188 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:44.643810 4188 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:44.644261 4026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce28976f0e640348fdb967365ae338c in term 1.
I20260619 20:48:44.644392 4136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cce28976f0e640348fdb967365ae338c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.644474 4136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:44.645795 4192 sys_catalog.cc:455] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce28976f0e640348fdb967365ae338c. Latest consensus state: current_term: 1 leader_uuid: "cce28976f0e640348fdb967365ae338c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.645874 4192 sys_catalog.cc:458] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:44.646867 4172 catalog_manager.cc:1347] Generated new cluster ID: e2f9ee469b49473d820770ffaa5d13bc
I20260619 20:48:44.646931 4172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:44.640565 4177 sys_catalog.cc:455] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.647136 4177 sys_catalog.cc:458] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:44.654593 4150 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cce28976f0e640348fdb967365ae338c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.654717 4150 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:44.655376 4177 sys_catalog.cc:455] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cce28976f0e640348fdb967365ae338c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce28976f0e640348fdb967365ae338c" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 35415 } } peers { permanent_uuid: "f510183729d64f0ca51b7d7f78454968" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 33595 } } peers { permanent_uuid: "6ee4c2c4fe044092ac42962ca3a58094" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 36433 } } }
I20260619 20:48:44.655501 4177 sys_catalog.cc:458] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:44.657089 4193 mvcc.cc:205] Tried to move back new op lower bound from 7298671102471090176 to 7298671102353764352. Current Snapshot: MvccSnapshot[applied={T|T < 7298671102471090176}]
I20260619 20:48:44.659862 4172 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:44.660668 4172 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:44.673477 4172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cce28976f0e640348fdb967365ae338c: Generated new TSK 0
I20260619 20:48:44.673712 4172 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:48:44.699097 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.697707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.13.195.125:33595)
0619 20:48:44.697781 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.697784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.698410 (+ 626us) server_negotiation.cc:407] Connection header received
0619 20:48:44.698534 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.698535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.698547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.698575 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.699021 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53080: BlockingRecv error: recv got EOF from 127.0.0.1:53080 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 20:48:44.699860 3980 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.698130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.13.195.126:35415)
0619 20:48:44.698170 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.698172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.699341 (+ 1169us) server_negotiation.cc:407] Connection header received
0619 20:48:44.699440 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.699441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.699453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.699483 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.699794 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39464: BlockingRecv error: recv got EOF from 127.0.0.1:39464 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:48:44.700862 4134 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.700216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.13.195.124:36433)
0619 20:48:44.700255 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.700257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.700390 (+ 133us) server_negotiation.cc:407] Connection header received
0619 20:48:44.700481 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.700482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.700492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.700520 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.700808 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40498: BlockingRecv error: recv got EOF from 127.0.0.1:40498 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:48:44.703173 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.702359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53092 (local address 127.13.195.125:33595)
0619 20:48:44.702394 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.702395 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.702659 (+ 264us) server_negotiation.cc:407] Connection header received
0619 20:48:44.702766 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.702767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.702786 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.702814 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.703125 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53092: BlockingRecv error: recv got EOF from 127.0.0.1:53092 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 20:48:44.706140 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.705374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.13.195.125:33595)
0619 20:48:44.705406 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.705408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.705650 (+ 242us) server_negotiation.cc:407] Connection header received
0619 20:48:44.705743 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.705743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.705754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.705782 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.706097 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53106: BlockingRecv error: recv got EOF from 127.0.0.1:53106 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 20:48:44.709865 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.709136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.13.195.125:33595)
0619 20:48:44.709189 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.709190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.709411 (+ 221us) server_negotiation.cc:407] Connection header received
0619 20:48:44.709503 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.709504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.709513 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.709541 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.709827 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53120: BlockingRecv error: recv got EOF from 127.0.0.1:53120 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 20:48:44.715790 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.715003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.13.195.125:33595)
0619 20:48:44.715041 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.715043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.715283 (+ 240us) server_negotiation.cc:407] Connection header received
0619 20:48:44.715376 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.715377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.715387 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.715414 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.715741 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53130: BlockingRecv error: recv got EOF from 127.0.0.1:53130 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:48:44.726101 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.725142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.13.195.125:33595)
0619 20:48:44.725185 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.725187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.725508 (+ 321us) server_negotiation.cc:407] Connection header received
0619 20:48:44.725621 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.725622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.725634 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.725663 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.726044 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53142: BlockingRecv error: recv got EOF from 127.0.0.1:53142 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 20:48:44.744463 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.743483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.13.195.125:33595)
0619 20:48:44.743530 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.743532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.743850 (+ 318us) server_negotiation.cc:407] Connection header received
0619 20:48:44.743972 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.743974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.743986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.744017 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.744403 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53150: BlockingRecv error: recv got EOF from 127.0.0.1:53150 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 20:48:44.778717 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.777776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.13.195.125:33595)
0619 20:48:44.777822 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.777824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.778113 (+ 289us) server_negotiation.cc:407] Connection header received
0619 20:48:44.778214 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.778215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.778226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.778255 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.778655 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53156: BlockingRecv error: recv got EOF from 127.0.0.1:53156 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 20:48:44.844802 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.844084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.13.195.125:33595)
0619 20:48:44.844173 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.844175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.844230 (+ 55us) server_negotiation.cc:407] Connection header received
0619 20:48:44.844285 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.844286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.844299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.844363 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.844732 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53162: BlockingRecv error: recv got EOF from 127.0.0.1:53162 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 20:48:44.974898 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:44.974105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.13.195.125:33595)
0619 20:48:44.974188 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:44.974190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:44.974301 (+ 111us) server_negotiation.cc:407] Connection header received
0619 20:48:44.974356 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:44.974357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:44.974368 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:44.974498 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:44.974844 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53172: BlockingRecv error: recv got EOF from 127.0.0.1:53172 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 20:48:45.232764 4057 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:45.232137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.13.195.125:33595)
0619 20:48:45.232199 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:45.232201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:45.232232 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:48:45.232320 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:45.232321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:45.232333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:45.232390 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:45.232712 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53174: BlockingRecv error: recv got EOF from 127.0.0.1:53174 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:48:45.591671 4147 catalog_manager.cc:1259] Loaded cluster ID: e2f9ee469b49473d820770ffaa5d13bc
I20260619 20:48:45.591779 4147 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: loading cluster ID for follower catalog manager: success
I20260619 20:48:45.592607 4147 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: acquiring CA information for follower catalog manager: success
I20260619 20:48:45.592921 4147 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ee4c2c4fe044092ac42962ca3a58094: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:48:45.644280 4188 catalog_manager.cc:1259] Loaded cluster ID: e2f9ee469b49473d820770ffaa5d13bc
I20260619 20:48:45.644373 4188 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: loading cluster ID for follower catalog manager: success
I20260619 20:48:45.645239 4188 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: acquiring CA information for follower catalog manager: success
I20260619 20:48:45.645578 4188 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f510183729d64f0ca51b7d7f78454968: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:48:45.748479 4261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:45.745902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.13.195.125:33595)
0619 20:48:45.745997 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:45.745999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:45.746037 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:48:45.746103 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:45.746104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:45.746153 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:45.746233 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:45.746413 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.746925 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.747744 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.747935 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.747986 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:45.748165 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:45.748282 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:45.748341 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:45.748393 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 20:48:45.748410 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":50,"threads_started":1}
I20260619 20:48:45.752890 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:45.750412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.13.195.124:36433)
0619 20:48:45.750577 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:45.750579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:45.750620 (+ 41us) server_negotiation.cc:407] Connection header received
0619 20:48:45.750654 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:45.750655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:45.750688 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:45.750752 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:45.750952 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.751426 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.752185 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.752366 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.752416 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:45.752547 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:45.752652 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:45.752709 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:45.752807 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 20:48:45.752827 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260619 20:48:45.757635 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:45.754755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.13.195.124:36433)
0619 20:48:45.754819 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:45.754821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:45.754934 (+ 113us) server_negotiation.cc:407] Connection header received
0619 20:48:45.755007 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:45.755008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:45.755017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:45.755079 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:45.755226 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.755744 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.756706 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.756995 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.757059 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:45.757220 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:45.757251 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:45.757306 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:45.757535 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 20:48:45.757557 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:48:45.757848 4261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:45.754850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.13.195.125:33595)
0619 20:48:45.754893 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:45.754895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:45.755131 (+ 236us) server_negotiation.cc:407] Connection header received
0619 20:48:45.755156 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:45.755157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:45.755167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:45.755217 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:45.755368 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.756087 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.756966 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.757245 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.757307 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:45.757449 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:45.757483 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:45.757530 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:45.757672 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 20:48:45.757695 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:45.758319 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:45.754759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42488 (local address 127.13.195.126:35415)
0619 20:48:45.754922 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:45.754924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:45.754932 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:45.754972 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:45.754973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:45.755026 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:45.755135 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:45.755286 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.756063 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.757001 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:45.757318 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:45.757371 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:45.757590 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:45.757929 (+ 339us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:45.758033 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:45.758206 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 20:48:45.758235 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
I20260619 20:48:45.760125 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3909
I20260619 20:48:45.765432 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 3984
I20260619 20:48:45.770208 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 4060
2026-06-19T20:48:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2853 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:48:45 dist-test-slave-n2td krb5kdc[4278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:48:45 dist-test-slave-n2td krb5kdc[4278](info): set up 2 sockets
Jun 19 20:48:45 dist-test-slave-n2td krb5kdc[4278](info): commencing operation
krb5kdc: starting...
W20260619 20:48:46.817648 14093 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/test-user.keytab.
Jun 19 20:48:46 dist-test-slave-n2td krb5kdc[4278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:48:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.195.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
Entry for principal kudu/127.13.195.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab.
I20260619 20:48:46.937872 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:38027
--webserver_interface=127.13.195.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:38027,127.13.195.125:43469,127.13.195.124:37607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:48:47.018647 4291 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:48:47.018898 4291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:47.018932 4291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:47.018975 4291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:47.020164 4291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:47.020238 4291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:47.020262 4291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:47.020285 4291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:47.020313 4291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:47.021783 4291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:38027,127.13.195.125:43469,127.13.195.124:37607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.126.keytab
--principal=kudu/127.13.195.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.126:38027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.195.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:47.022070 4291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:47.022302 4291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:47.027076 4299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:47.027096 4297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:47.027077 4296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:47.027561 4291 server_base.cc:1034] running on GCE node
I20260619 20:48:47.027875 4291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:47.028216 4291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:47.029407 4291 hybrid_clock.cc:648] HybridClock initialized: now 1781902127029371 us; error 65 us; skew 500 ppm
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:47.032006 4291 init.cc:377] Logged in from keytab as kudu/127.13.195.126@KRBTEST.COM (short username kudu)
I20260619 20:48:47.032456 4291 webserver.cc:460] Webserver started at http://127.13.195.126:36185/ using document root <none> and password file <none>
I20260619 20:48:47.032653 4291 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:47.032693 4291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:47.032779 4291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:47.033790 4291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data/instance:
uuid: "6206f8a1f66d4765b8e07151c2516611"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "47fc293ab0bbc1a9c1bab119f1c6a0b6"
server_key_iv: "af9db11558543a3c8359f4573ad27445"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.034096 4291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/wal/instance:
uuid: "6206f8a1f66d4765b8e07151c2516611"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "47fc293ab0bbc1a9c1bab119f1c6a0b6"
server_key_iv: "af9db11558543a3c8359f4573ad27445"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.035741 4291 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260619 20:48:47.036460 4306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:47.036669 4291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:47.036761 4291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/wal
uuid: "6206f8a1f66d4765b8e07151c2516611"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "47fc293ab0bbc1a9c1bab119f1c6a0b6"
server_key_iv: "af9db11558543a3c8359f4573ad27445"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.036823 4291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:47.064610 4291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:47.066535 4291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:47.066648 4291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:47.070716 4291 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.126:38027
I20260619 20:48:47.070781 4359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.126:38027 every 8 connection(s)
I20260619 20:48:47.071123 4291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/data/info.pb
I20260619 20:48:47.071986 4360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:47.072862 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 4291
I20260619 20:48:47.072947 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-0/wal/instance
I20260619 20:48:47.073150 14093 external_mini_cluster.cc:1428] Setting key 6dd603109a91eb83eb909b33dbec8a9c
I20260619 20:48:47.073701 4360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.125@KRBTEST.COM" created.
I20260619 20:48:47.083308 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.074465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:38027 (local address 127.0.0.1:39804)
0619 20:48:47.074618 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.074662 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.075133 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.075433 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.075436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.075716 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.075867 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.075872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.076965 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.076966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.077630 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.077632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.078140 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.078141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.078208 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.078780 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.078788 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.080134 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.081851 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.081856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.081858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.082046 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.082323 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.082324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.082325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.082407 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.082860 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.082867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.082930 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.083011 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.083142 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260619 20:48:47.083563 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.074506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.13.195.126:38027)
0619 20:48:47.074691 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.074696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.074737 (+ 41us) server_negotiation.cc:407] Connection header received
0619 20:48:47.075303 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.075306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.075332 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.075428 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.075971 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.076854 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.077723 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.078037 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.078169 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.080232 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.080240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.080240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.080247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.081723 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.082137 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.082140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.082141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.082220 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.082516 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.082517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.082518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.082768 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.082870 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.083084 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.083234 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
I20260619 20:48:47.084210 4360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:47.084568 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.084441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34720)
0619 20:48:47.084498 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.084546 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 20:48:47.084672 4307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
I20260619 20:48:47.085119 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.085021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34722)
0619 20:48:47.085072 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.085098 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 20:48:47.085238 4360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:43469: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab.
Entry for principal kudu/127.13.195.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab.
I20260619 20:48:47.098156 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:43469
--webserver_interface=127.13.195.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab
--principal=kudu/127.13.195.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:38027,127.13.195.125:43469,127.13.195.124:37607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:48:47.134500 4360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } attempt: 1
I20260619 20:48:47.134964 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.134828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34734)
0619 20:48:47.134896 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.134935 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:48:47.135430 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.135265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34738)
0619 20:48:47.135338 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.135400 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 20:48:47.135568 4360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:43469: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
W20260619 20:48:47.178978 4367 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:48:47.179194 4367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:47.179211 4367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:47.179231 4367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:47.180323 4367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:47.180357 4367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:47.180371 4367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:47.180383 4367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:47.180395 4367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:47.181773 4367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:38027,127.13.195.125:43469,127.13.195.124:37607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.125.keytab
--principal=kudu/127.13.195.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.125:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.195.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:47.181968 4367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:47.182214 4367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:47.186966 4373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:47.187003 4376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:47.186975 4374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:47.187141 4367 server_base.cc:1034] running on GCE node
I20260619 20:48:47.187464 4367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:47.187748 4367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:47.188894 4367 hybrid_clock.cc:648] HybridClock initialized: now 1781902127188873 us; error 35 us; skew 500 ppm
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:47.191395 4367 init.cc:377] Logged in from keytab as kudu/127.13.195.125@KRBTEST.COM (short username kudu)
I20260619 20:48:47.191857 4367 webserver.cc:460] Webserver started at http://127.13.195.125:46859/ using document root <none> and password file <none>
I20260619 20:48:47.192088 4367 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:47.192133 4367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:47.192309 4367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:47.193360 4367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data/instance:
uuid: "7e58b7069c0447ec9670fe21ff133e0b"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "b3cf5009f55df4e891a170892951b8ab"
server_key_iv: "b64440a7668babc6bd4e20195c4e7fa8"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.193717 4367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/wal/instance:
uuid: "7e58b7069c0447ec9670fe21ff133e0b"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "b3cf5009f55df4e891a170892951b8ab"
server_key_iv: "b64440a7668babc6bd4e20195c4e7fa8"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.195112 4367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:48:47.195832 4383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:47.196002 4367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:47.196103 4367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/wal
uuid: "7e58b7069c0447ec9670fe21ff133e0b"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "b3cf5009f55df4e891a170892951b8ab"
server_key_iv: "b64440a7668babc6bd4e20195c4e7fa8"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.196177 4367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:47.203706 4360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } attempt: 2
I20260619 20:48:47.204191 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.204053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34740)
0619 20:48:47.204121 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.204161 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 20:48:47.204523 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.204441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34752)
0619 20:48:47.204484 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.204507 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:48:47.204630 4360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.125:43469: Network error: Client connection negotiation failed: client connection to 127.13.195.125:43469: connect: Connection refused (error 111)
I20260619 20:48:47.206883 4367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:47.208665 4367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:47.208820 4367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:47.213099 4367 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.125:43469
I20260619 20:48:47.213127 4435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.125:43469 every 8 connection(s)
I20260619 20:48:47.213478 4367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/data/info.pb
I20260619 20:48:47.214336 4436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:47.215873 4436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:47.222755 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 4367
I20260619 20:48:47.222846 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-1/wal/instance
I20260619 20:48:47.223104 14093 external_mini_cluster.cc:1428] Setting key 99e57a23df77dec2bb8b5aa3037b9281
I20260619 20:48:47.224875 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.216590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.13.195.126:38027)
0619 20:48:47.216659 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.216662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.216885 (+ 223us) server_negotiation.cc:407] Connection header received
0619 20:48:47.217339 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.217340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.217353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.217406 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.218004 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.218532 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.219456 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.219641 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.219694 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.221710 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.221711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.221712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.221718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.223318 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.223813 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.223814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.223814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.223864 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.224131 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.224131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.224132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.224232 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.224281 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.224657 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.224754 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:48:47.224978 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.216556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:38027 (local address 127.0.0.1:39808)
0619 20:48:47.216764 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.216802 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.217258 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.217484 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.217487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.217739 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.217932 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.217935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.218613 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.218613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.219386 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.219388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.219714 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.219715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.219814 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.220283 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.220291 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.221627 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.223394 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.223400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.223402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.223735 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.223928 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.223929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.223929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.224071 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.224294 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.224300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.224465 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.224620 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.224785 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
I20260619 20:48:47.225666 4436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.195.124@KRBTEST.COM; defaulting to no policy
I20260619 20:48:47.232887 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.225955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34764)
0619 20:48:47.226017 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.226028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.226087 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.226320 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.226321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.226508 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.226562 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.226564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.227426 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.227427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.227922 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.227924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.228421 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.228422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.228488 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.228907 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.228912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.230080 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.231843 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.231844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.231844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.232003 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.232234 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.232235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.232235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.232286 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.232563 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.232564 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.232598 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.232673 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.232775 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:48:47.232952 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.226029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.13.195.125:43469)
0619 20:48:47.226180 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.226185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.226205 (+ 20us) server_negotiation.cc:407] Connection header received
0619 20:48:47.226230 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.226231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.226241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.226320 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.226637 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.227322 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.228002 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.228335 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.228414 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.230178 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.230185 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.230185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.230191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.231766 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.232099 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.232102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.232103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.232159 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.232353 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.232354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.232354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.232493 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.232601 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.232723 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.232827 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.195.124@KRBTEST.COM" created.
I20260619 20:48:47.233670 4436 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:47.234056 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.233907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34522)
0619 20:48:47.233972 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.234028 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 20:48:47.234237 4386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
I20260619 20:48:47.234704 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.234600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34538)
0619 20:48:47.234644 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.234672 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 20:48:47.234862 4436 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37607: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.195.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab.
Entry for principal kudu/127.13.195.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab.
I20260619 20:48:47.249876 14093 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhOhR3l/build/release/bin/kudu
/tmp/dist-test-taskhOhR3l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:37607
--webserver_interface=127.13.195.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab
--principal=kudu/127.13.195.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.195.84:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.195.126:38027,127.13.195.125:43469,127.13.195.124:37607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:48:47.284113 4436 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } attempt: 1
I20260619 20:48:47.284615 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.284479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34544)
0619 20:48:47.284541 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.284583 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:48:47.285019 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.284917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34550)
0619 20:48:47.284969 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.284993 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:48:47.285106 4436 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37607: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
I20260619 20:48:47.295785 4360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } attempt: 3
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:48:47.303056 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.296173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34774)
0619 20:48:47.296240 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.296253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.296310 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.296473 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.296474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.296636 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.296701 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.296703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.297362 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.297364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.297895 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.297897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.298306 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.298307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.298386 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.298933 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.298938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.300259 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.301940 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.301940 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.301941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.302109 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.302359 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.302360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.302360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.302495 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.302786 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.302787 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.302826 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.302880 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.302966 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:48:47.303141 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.296251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.13.195.125:43469)
0619 20:48:47.296310 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.296313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.296323 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:47.296373 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.296374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.296386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.296465 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.296792 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.297282 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.297998 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.298229 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.298309 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.300345 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.300346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.300347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.300355 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.301853 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.302178 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.302180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.302180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.302244 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.302573 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.302573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.302574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.302715 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.302788 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.302915 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.302992 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:48:47.303512 4360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:47.303858 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.303749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34566)
0619 20:48:47.303800 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.303830 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:48:47.304258 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.304142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34570)
0619 20:48:47.304204 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.304232 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 20:48:47.304389 4360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37607: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
W20260619 20:48:47.333726 4443 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:48:47.333962 4443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:48:47.333982 4443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:48:47.334004 4443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:48:47.335346 4443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:48:47.335394 4443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:48:47.335412 4443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:48:47.335457 4443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:48:47.335486 4443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:48:47.335530 4360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } attempt: 1
I20260619 20:48:47.335994 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.335840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34580)
0619 20:48:47.335918 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.335963 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 20:48:47.336417 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.336327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34586)
0619 20:48:47.336374 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.336399 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:48:47.336541 4360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37607: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
I20260619 20:48:47.337514 4443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.195.84:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.195.126:38027,127.13.195.125:43469,127.13.195.124:37607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/krb5kdc/kudu_127.13.195.124.keytab
--principal=kudu/127.13.195.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.195.124:37607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.195.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:59:35 UTC on ccycloud.quasar-bezeai.root.comops.site
build id 4456677
I20260619 20:48:47.337739 4443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:48:47.337930 4443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:48:47.343096 4449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:47.343111 4451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:48:47.343102 4448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:48:47.343163 4443 server_base.cc:1034] running on GCE node
I20260619 20:48:47.343559 4443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:48:47.343825 4443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:48:47.344973 4443 hybrid_clock.cc:648] HybridClock initialized: now 1781902127344953 us; error 36 us; skew 500 ppm
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:48:47.347775 4443 init.cc:377] Logged in from keytab as kudu/127.13.195.124@KRBTEST.COM (short username kudu)
I20260619 20:48:47.348377 4443 webserver.cc:460] Webserver started at http://127.13.195.124:42237/ using document root <none> and password file <none>
I20260619 20:48:47.348634 4443 fs_manager.cc:359] Metadata directory not provided
I20260619 20:48:47.348716 4443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:48:47.348877 4443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:48:47.350039 4443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data/instance:
uuid: "7996a84a8ec7403c8ff928e99c2edc0f"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "2add511268795302175a71da4f2274f2"
server_key_iv: "f6f27d86c5e4e0ea42e9461140f10994"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.350481 4443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/wal/instance:
uuid: "7996a84a8ec7403c8ff928e99c2edc0f"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "2add511268795302175a71da4f2274f2"
server_key_iv: "f6f27d86c5e4e0ea42e9461140f10994"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.351984 4443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:48:47.352717 4458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:48:47.352891 4443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:48:47.353001 4443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data,/tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/wal
uuid: "7996a84a8ec7403c8ff928e99c2edc0f"
format_stamp: "Formatted at 2026-06-19 20:48:47 on dist-test-slave-n2td"
server_key: "2add511268795302175a71da4f2274f2"
server_key_iv: "f6f27d86c5e4e0ea42e9461140f10994"
server_key_version: "encryptionkey@0"
I20260619 20:48:47.353086 4443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:48:47.353246 4436 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } attempt: 2
I20260619 20:48:47.353633 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.353512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34602)
0619 20:48:47.353566 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.353606 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:48:47.353965 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.353868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34604)
0619 20:48:47.353913 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.353936 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 20:48:47.354050 4436 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.195.124:37607: Network error: Client connection negotiation failed: client connection to 127.13.195.124:37607: connect: Connection refused (error 111)
I20260619 20:48:47.383761 4443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:48:47.386085 4443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:48:47.386250 4443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:48:47.390369 4443 rpc_server.cc:307] RPC server started. Bound to: 127.13.195.124:37607
I20260619 20:48:47.390470 4511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.195.124:37607 every 8 connection(s)
I20260619 20:48:47.390864 4443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/data/info.pb
I20260619 20:48:47.391686 4512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:48:47.393241 4512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:47.394889 14093 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhOhR3l/build/release/bin/kudu as pid 4443
I20260619 20:48:47.395000 14093 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhOhR3l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901857247738-14093-0/minicluster-data/master-2/wal/instance
I20260619 20:48:47.395210 14093 external_mini_cluster.cc:1428] Setting key 00f77b38425379283d705bf065085ed8
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:48:47.402887 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.393974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.13.195.126:38027)
0619 20:48:47.394081 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.394083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.394250 (+ 167us) server_negotiation.cc:407] Connection header received
0619 20:48:47.394799 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.394801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.394814 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.394864 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.395773 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.396255 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.397236 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.397427 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.397528 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.399839 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.399841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.399841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.399849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.401348 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.401829 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.401831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.401832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.401887 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.402179 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.402180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.402180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.402315 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.402390 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.402697 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.402773 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 20:48:47.402917 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.393931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.126:38027 (local address 127.0.0.1:39810)
0619 20:48:47.394109 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.394151 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.394684 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.394931 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.394936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.395283 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.395654 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.395661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.396349 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.396350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.397130 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.397132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.397535 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.397535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.397673 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.398201 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.398211 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.399752 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.401445 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.401452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.401454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.401760 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.401957 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.401957 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.401957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.402110 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.402387 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.402393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.402562 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.402653 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.402763 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260619 20:48:47.403488 4512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:47.404496 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.395559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.13.195.124:37607)
0619 20:48:47.395730 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.395734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.395854 (+ 120us) server_negotiation.cc:407] Connection header received
0619 20:48:47.395901 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.395904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.395926 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.395999 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.396606 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.397300 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.398181 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.398636 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.398709 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.401157 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.401164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.401165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.401171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.403100 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.403658 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.403661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.403661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.403716 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.403925 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.403926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.403926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.404098 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.404232 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.404323 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.404397 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":113,"threads_started":1}
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.126@KRBTEST.COM
I20260619 20:48:47.410066 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.403744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.125:43469 (local address 127.0.0.1:34780)
0619 20:48:47.403786 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.403795 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.403823 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.404012 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.404013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.404122 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.404176 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.404177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.405022 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.405023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.405664 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.405665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.406070 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.406071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.406129 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.406484 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.406489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.407564 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.409057 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.409057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.409058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.409219 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.409438 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.409439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.409440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.409497 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.409780 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.409780 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.409812 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.409890 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.409976 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 20:48:47.410099 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.403825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.13.195.125:43469)
0619 20:48:47.403896 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.403898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.403907 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:48:47.403932 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.403932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.403943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.403996 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.404258 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.404934 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.405739 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.406005 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.406054 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.407648 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.407650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.407650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.407658 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.408978 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.409289 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.409290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.409290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.409345 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.409564 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.409566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.409566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.409708 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.409784 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.409938 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.410009 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:48:47.410691 4512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:48:47.411696 4360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } attempt: 2
I20260619 20:48:47.412818 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.405264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39818 (local address 127.13.195.126:38027)
0619 20:48:47.405302 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.405304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.405314 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:47.405346 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.405347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.405355 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.405397 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.405840 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.406416 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.407120 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.407309 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.407395 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.409860 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.409861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.409862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.409871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.411538 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.411877 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.411879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.411879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.411952 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.412147 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.412148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.412149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.412252 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.412447 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.412650 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.412717 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.124@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.126@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.195.125@KRBTEST.COM
I20260619 20:48:47.418548 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.410933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34614)
0619 20:48:47.410969 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.410981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.411024 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.411137 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.411138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.411259 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.411291 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.411293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.412038 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.412038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.412707 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.412708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.413181 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.413182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.413222 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.413601 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.413606 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.414969 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.417379 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.417380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.417380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.417601 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.417799 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.417800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.417800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.417865 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.418154 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.418155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.418195 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.418312 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.418399 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 20:48:47.418915 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.410976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.13.195.124:37607)
0619 20:48:47.411006 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.411008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.411016 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:48:47.411079 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.411079 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.411087 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.411124 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.411361 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.411959 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.412784 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.413093 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.413144 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.415056 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.415057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.415057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.415073 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.417297 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.417660 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.417661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.417661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.417718 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.417936 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.417936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.417937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.418047 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.418718 (+ 671us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.418760 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.418835 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 20:48:47.419706 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.411908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34616)
0619 20:48:47.411949 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.411960 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.411995 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.412317 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.412318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.412473 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.412510 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.412512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.413475 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.413477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.413984 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.413986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.414497 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.414498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.414559 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.414984 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.414989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.416195 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.418382 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.418383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.418383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.418588 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.419098 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.419100 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.419100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.419163 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.419439 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.419440 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.419481 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.419532 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.419625 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 20:48:47.420125 4517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.412067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34616 (local address 127.13.195.124:37607)
0619 20:48:47.412199 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.412201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.412207 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:48:47.412223 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.412224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.412232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.412301 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.412593 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.413310 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.414071 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.414366 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.414464 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.416297 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.416298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.416299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.416305 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.418283 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.418692 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.418693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.418694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.418751 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.419235 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.419235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.419236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.419356 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.419910 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.419978 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.420052 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260619 20:48:47.420524 4512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: Bootstrap starting.
I20260619 20:48:47.421391 4512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:47.421518 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.413381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.13.195.125:43469)
0619 20:48:47.413420 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.413422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.413429 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:48:47.413461 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.413462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.413471 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.413518 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.414143 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.414814 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.415678 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.415895 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.415979 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.418585 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.418586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.418587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.418594 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.420284 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.420647 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.420648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.420648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.420706 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.420915 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.420917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.420917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.421110 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.421284 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.421335 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.421410 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:47.421754 4512 log.cc:824] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:47.422612 4512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: No bootstrap required, opened a new log
I20260619 20:48:47.423414 14093 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:48:47.423746 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611: Bootstrap starting.
I20260619 20:48:47.424449 4360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:47.424747 4360 log.cc:824] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:47.424739 4512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
I20260619 20:48:47.424893 4512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7996a84a8ec7403c8ff928e99c2edc0f, State: Initialized, Role: FOLLOWER
I20260619 20:48:47.424978 4512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
I20260619 20:48:47.425277 4512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:47.424427 4333 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6206f8a1f66d4765b8e07151c2516611"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39818
I20260619 20:48:47.425239 4520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.425501 4520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:47.425540 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611: No bootstrap required, opened a new log
I20260619 20:48:47.426941 4360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
W20260619 20:48:47.427002 4531 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:47.427075 4531 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:47.427079 4360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6206f8a1f66d4765b8e07151c2516611, State: Initialized, Role: FOLLOWER
I20260619 20:48:47.427163 4360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
I20260619 20:48:47.427412 4533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.427479 4360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:47.427497 4533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:48:47.429338 4544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:47.429387 4544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:47.427441 4443 master_runner.cc:186] Error getting master registration for 127.13.195.126:38027: OK, instance_id { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" instance_seqno: 1781902127069568 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:48:47.430686 4443 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:48:47.430774 4333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 20:48:47.430827 4333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:47.431656 4333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
I20260619 20:48:47.431879 4333 leader_election.cc:288] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [CANDIDATE]: Term 1 election: Requested vote from peers 7e58b7069c0447ec9670fe21ff133e0b (127.13.195.125:43469), 7996a84a8ec7403c8ff928e99c2edc0f (127.13.195.124:37607)
I20260619 20:48:47.429996 4291 master_runner.cc:186] Error getting master registration for 127.13.195.125:43469: OK, instance_id { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" instance_seqno: 1781902127211856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:48:47.432986 4291 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:48:47.433295 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6206f8a1f66d4765b8e07151c2516611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7996a84a8ec7403c8ff928e99c2edc0f"
I20260619 20:48:47.433426 4486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:47.434887 4486 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6206f8a1f66d4765b8e07151c2516611 in term 1.
I20260619 20:48:47.435132 4309 leader_election.cc:302] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6206f8a1f66d4765b8e07151c2516611, 7996a84a8ec7403c8ff928e99c2edc0f; no voters:
I20260619 20:48:47.435238 4533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:48:47.435359 4533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [term 1 LEADER]: Becoming Leader. State: Replica: 6206f8a1f66d4765b8e07151c2516611, State: Running, Role: LEADER
I20260619 20:48:47.435459 4533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
I20260619 20:48:47.435891 4533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6206f8a1f66d4765b8e07151c2516611. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.435992 4533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:47.436338 4555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:48:47.437216 4555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:48:47.441931 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.433154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.13.195.125:43469)
0619 20:48:47.433196 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.433198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.433615 (+ 417us) server_negotiation.cc:407] Connection header received
0619 20:48:47.433720 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.433721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.433734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.433763 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.434611 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.435350 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.436680 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.436989 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.437038 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.438853 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.438854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.438855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.438864 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.440674 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.441062 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.441063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.441063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.441118 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.441295 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.441295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.441295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.441423 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.441494 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.441752 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.441831 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:48:47.445173 4436 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } attempt: 3
I20260619 20:48:47.449141 4486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [term 1 FOLLOWER]: Refusing update from remote peer 6206f8a1f66d4765b8e07151c2516611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:47.449476 4553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:47.450767 4520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6206f8a1f66d4765b8e07151c2516611. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.450865 4520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:47.451282 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.434617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.13.195.126:38027)
0619 20:48:47.434654 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.434656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.439019 (+ 4363us) server_negotiation.cc:407] Connection header received
0619 20:48:47.439210 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.439211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.439222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.439252 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.443707 (+ 4455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.444462 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.445774 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.446071 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.446109 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.447505 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.447506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.447507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.447514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.449380 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.449980 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.449981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.449981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.450283 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.450526 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.450526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.450527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.450652 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.450750 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.451101 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.451179 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 20:48:47 dist-test-slave-n2td krb5kdc[4278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781902127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.195.125@KRBTEST.COM for kudu/127.13.195.124@KRBTEST.COM
I20260619 20:48:47.452468 4517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.434505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.13.195.124:37607)
0619 20:48:47.434548 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.434550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.439232 (+ 4682us) server_negotiation.cc:407] Connection header received
0619 20:48:47.439333 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.439334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.439345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.439374 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.440561 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.441322 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.442734 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.443089 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.443136 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.445111 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.445112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.445113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.445119 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.449390 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.451235 (+ 1845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.451236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.451236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.451305 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.451651 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.451652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.451653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.451826 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.451911 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.452261 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.452354 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:48:47.455255 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.445416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.195.124:37607 (local address 127.0.0.1:34628)
0619 20:48:47.445461 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:48:47.445471 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:48:47.445508 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:48:47.445709 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:48:47.445711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:48:47.445888 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:48:47.445925 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.445927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.446902 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.446903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.447776 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:48:47.447779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.449810 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.449811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:48:47.449891 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.450261 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:48:47.450276 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:48:47.451863 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:48:47.454036 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.454037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.454037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.454260 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.454547 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:48:47.454548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:48:47.454548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:48:47.454629 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:48:47.454937 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:48:47.454937 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:48:47.454988 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 20:48:47.455056 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:48:47.455156 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:48:47.455295 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:47.445521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34628 (local address 127.13.195.124:37607)
0619 20:48:47.445564 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.445566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.445573 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:48:47.445592 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.445593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.445602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.445653 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.446014 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.446790 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.448299 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:47.449686 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:47.449744 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:47.451954 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:48:47.451955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:48:47.451955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:48:47.451962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:48:47.453961 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.454351 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.454352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.454353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.454461 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:48:47.454710 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:48:47.454712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:48:47.454712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:48:47.454867 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:48:47.454930 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:47.455148 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 20:48:47.455212 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:48:47.455417 4553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.455529 4553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:47.456022 4520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.456106 4520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:47.456171 4556 mvcc.cc:205] Tried to move back new op lower bound from 7298671114005397504 to 7298671113976336384. Current Snapshot: MvccSnapshot[applied={T|T < 7298671114005397504}]
I20260619 20:48:47.456769 4557 mvcc.cc:205] Tried to move back new op lower bound from 7298671114005397504 to 7298671113976336384. Current Snapshot: MvccSnapshot[applied={T|T < 7298671114005397504}]
I20260619 20:48:47.457175 4436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: Bootstrap starting.
I20260619 20:48:47.457890 4436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: Neither blocks nor log segments found. Creating new log.
I20260619 20:48:47.458187 4436 log.cc:824] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:48:47.463160 4436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: No bootstrap required, opened a new log
I20260619 20:48:47.465274 4436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
I20260619 20:48:47.465415 4436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e58b7069c0447ec9670fe21ff133e0b, State: Initialized, Role: FOLLOWER
I20260619 20:48:47.465525 4436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } }
I20260619 20:48:47.465876 4436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:48:47.465961 4410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6206f8a1f66d4765b8e07151c2516611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7e58b7069c0447ec9670fe21ff133e0b"
I20260619 20:48:47.466066 4410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:48:47.467093 4410 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6206f8a1f66d4765b8e07151c2516611 in term 1.
I20260619 20:48:47.467337 4409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [term 1 FOLLOWER]: Refusing update from remote peer 6206f8a1f66d4765b8e07151c2516611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:48:47.468544 4533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:48:47.469995 4577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6206f8a1f66d4765b8e07151c2516611. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.470149 4577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:47.470234 4577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.470321 4577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:47.470832 4555 catalog_manager.cc:1347] Generated new cluster ID: c3d6d45c9f4146ddbf0bbff629cc5e84
I20260619 20:48:47.470892 4555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:48:47.473845 4575 mvcc.cc:205] Tried to move back new op lower bound from 7298671114005397504 to 7298671113976336384. Current Snapshot: MvccSnapshot[applied={T|T < 7298671114005397504}]
W20260619 20:48:47.478962 4573 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:48:47.479032 4573 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:48:47.479175 4562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.479255 4562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:48:47.479732 4533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6206f8a1f66d4765b8e07151c2516611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6206f8a1f66d4765b8e07151c2516611" member_type: VOTER last_known_addr { host: "127.13.195.126" port: 38027 } } peers { permanent_uuid: "7e58b7069c0447ec9670fe21ff133e0b" member_type: VOTER last_known_addr { host: "127.13.195.125" port: 43469 } } peers { permanent_uuid: "7996a84a8ec7403c8ff928e99c2edc0f" member_type: VOTER last_known_addr { host: "127.13.195.124" port: 37607 } } }
I20260619 20:48:47.479823 4533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611 [sys.catalog]: This master's current role is: LEADER
I20260619 20:48:47.481171 4555 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:48:47.481879 4555 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:48:47.498369 4555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6206f8a1f66d4765b8e07151c2516611: Generated new TSK 0
I20260619 20:48:47.498636 4555 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:48:47.524570 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.523536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34792 (local address 127.13.195.125:43469)
0619 20:48:47.523583 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.523585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.523898 (+ 313us) server_negotiation.cc:407] Connection header received
0619 20:48:47.524004 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.524005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.524016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.524046 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.524497 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34792: BlockingRecv error: recv got EOF from 127.0.0.1:34792 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 20:48:47.526137 4516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.525200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34634 (local address 127.13.195.124:37607)
0619 20:48:47.525243 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.525245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.525670 (+ 425us) server_negotiation.cc:407] Connection header received
0619 20:48:47.525696 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.525697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.525707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.525751 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.526068 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34634: BlockingRecv error: recv got EOF from 127.0.0.1:34634 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 20:48:47.527035 4364 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.525403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.13.195.126:38027)
0619 20:48:47.525443 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.525445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.526574 (+ 1129us) server_negotiation.cc:407] Connection header received
0619 20:48:47.526603 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.526604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.526615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.526670 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.526972 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39840: BlockingRecv error: recv got EOF from 127.0.0.1:39840 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 20:48:47.529475 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.528683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.13.195.125:43469)
0619 20:48:47.528731 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.528733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.528946 (+ 213us) server_negotiation.cc:407] Connection header received
0619 20:48:47.529054 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.529055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.529076 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.529105 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.529425 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34804: BlockingRecv error: recv got EOF from 127.0.0.1:34804 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 20:48:47.532585 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.531802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34808 (local address 127.13.195.125:43469)
0619 20:48:47.531836 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.531838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.532109 (+ 271us) server_negotiation.cc:407] Connection header received
0619 20:48:47.532207 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.532208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.532219 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.532248 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.532536 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34808: BlockingRecv error: recv got EOF from 127.0.0.1:34808 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:48:47.536592 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.535788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.13.195.125:43469)
0619 20:48:47.535823 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.535825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.536098 (+ 273us) server_negotiation.cc:407] Connection header received
0619 20:48:47.536195 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.536195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.536206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.536235 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.536542 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34820: BlockingRecv error: recv got EOF from 127.0.0.1:34820 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 20:48:47.549830 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.548913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34830 (local address 127.13.195.125:43469)
0619 20:48:47.548956 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.548957 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.549255 (+ 298us) server_negotiation.cc:407] Connection header received
0619 20:48:47.549359 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.549360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.549371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.549402 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.549770 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34830: BlockingRecv error: recv got EOF from 127.0.0.1:34830 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 20:48:47.560103 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.559242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.13.195.125:43469)
0619 20:48:47.559282 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.559284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.559528 (+ 244us) server_negotiation.cc:407] Connection header received
0619 20:48:47.559645 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.559646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.559658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.559688 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.560052 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34832: BlockingRecv error: recv got EOF from 127.0.0.1:34832 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:48:47.578652 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.577673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.13.195.125:43469)
0619 20:48:47.577718 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.577721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.577986 (+ 265us) server_negotiation.cc:407] Connection header received
0619 20:48:47.578095 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.578097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.578109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.578142 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.578579 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34846: BlockingRecv error: recv got EOF from 127.0.0.1:34846 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 20:48:47.613207 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.612242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.13.195.125:43469)
0619 20:48:47.612289 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.612291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.612560 (+ 269us) server_negotiation.cc:407] Connection header received
0619 20:48:47.612689 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.612691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.612704 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.612736 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.613144 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34856: BlockingRecv error: recv got EOF from 127.0.0.1:34856 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 20:48:47.679728 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.678973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.13.195.125:43469)
0619 20:48:47.679052 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.679054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.679174 (+ 120us) server_negotiation.cc:407] Connection header received
0619 20:48:47.679234 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.679236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.679247 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.679306 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.679662 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34868: BlockingRecv error: recv got EOF from 127.0.0.1:34868 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 20:48:47.809731 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:47.808970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.13.195.125:43469)
0619 20:48:47.809043 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:47.809046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:47.809157 (+ 111us) server_negotiation.cc:407] Connection header received
0619 20:48:47.809227 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:47.809228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:47.809238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:47.809282 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:47.809657 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34882: BlockingRecv error: recv got EOF from 127.0.0.1:34882 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 20:48:48.067881 4440 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:48:48.067094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.13.195.125:43469)
0619 20:48:48.067162 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:48.067164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:48.067203 (+ 39us) server_negotiation.cc:407] Connection header received
0619 20:48:48.067275 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:48.067276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:48.067288 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:48.067398 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:48:48.067775 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34888: BlockingRecv error: recv got EOF from 127.0.0.1:34888 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:48:48.427603 4531 catalog_manager.cc:1259] Loaded cluster ID: c3d6d45c9f4146ddbf0bbff629cc5e84
I20260619 20:48:48.427678 4531 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: loading cluster ID for follower catalog manager: success
I20260619 20:48:48.428633 4531 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: acquiring CA information for follower catalog manager: success
I20260619 20:48:48.428952 4531 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7996a84a8ec7403c8ff928e99c2edc0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:48:48.479506 4573 catalog_manager.cc:1259] Loaded cluster ID: c3d6d45c9f4146ddbf0bbff629cc5e84
I20260619 20:48:48.479589 4573 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: loading cluster ID for follower catalog manager: success
I20260619 20:48:48.480485 4573 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: acquiring CA information for follower catalog manager: success
I20260619 20:48:48.480773 4573 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e58b7069c0447ec9670fe21ff133e0b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:48:48.583699 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:48.581074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.13.195.125:43469)
0619 20:48:48.581203 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:48.581205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:48.581263 (+ 58us) server_negotiation.cc:407] Connection header received
0619 20:48:48.581311 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:48.581311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:48.581362 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:48.581439 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:48.581582 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.582078 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.582832 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.583041 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.583108 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:48.583276 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:48.583399 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:48.583490 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:48.583513 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 20:48:48.583542 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":49,"threads_started":1}
I20260619 20:48:48.588232 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:48.585686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.13.195.124:37607)
0619 20:48:48.585812 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:48.585815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:48.585824 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:48:48.585910 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:48.585911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:48.585960 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:48.586046 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:48.586192 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.586702 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.587418 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.587639 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.587714 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:48.587877 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:48.587984 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:48.588069 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:48.588112 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 20:48:48.588132 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
I20260619 20:48:48.593235 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:48.590228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.13.195.124:37607)
0619 20:48:48.590271 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:48.590274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:48.590284 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:48:48.590372 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:48.590373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:48.590384 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:48.590499 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:48.590681 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.591436 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.592484 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.592733 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.592796 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:48.592954 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:48.592979 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:48.593019 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:48.593121 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 20:48:48.593149 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:48.593386 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:48.590062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34914 (local address 127.13.195.125:43469)
0619 20:48:48.590100 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:48.590102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:48.590457 (+ 355us) server_negotiation.cc:407] Connection header received
0619 20:48:48.590553 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:48.590555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:48.590566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:48.590618 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:48.590794 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.591244 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.592383 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.592699 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.592769 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:48.592947 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:48.592982 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:48.593062 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:48.593290 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 20:48:48.593315 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:48:48.593758 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:48:48.590062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39846 (local address 127.13.195.126:38027)
0619 20:48:48.590179 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 20:48:48.590181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:48:48.590301 (+ 120us) server_negotiation.cc:407] Connection header received
0619 20:48:48.590355 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:48:48.590356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:48:48.590515 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:48:48.590600 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:48:48.590732 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.591481 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.592317 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:48:48.592615 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:48:48.592688 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:48:48.592876 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:48:48.592987 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:48:48.593077 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:48:48.593132 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 20:48:48.593155 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260619 20:48:48.599069 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 4291
I20260619 20:48:48.604348 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 4367
I20260619 20:48:48.609148 14093 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhOhR3l/build/release/bin/kudu with pid 4443
2026-06-19T20:48:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2839 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5693 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271369 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST